Generalizing find-definition

Generalizing find-definition

[Jorgen Schaefer]

Hello!
Emacs by default uses etags.el to find definitions. This is great for C
and similar more static languages, but many more dynamic languages have
other ways of finding definitions. This has lead to major/minor modes
redefining M-. to a specialized "go to definition" function (e.g.
SLIME for CL, slime-nav for Emacs Lisp, Geiser, Cider, Elpy etc.). I
think it would be good if Emacs provided a generalized functionality
for M-. and related commands instead of having all these modes
re-implementing it.

If I get positive feedback on the following ideas, I'm happy to do the
necessary coding (once the git transition is done).


M-. is bound to a new command `find-definition', which primarily calls
the value of a new variable `find-definition-function' (by default a
wrapper around `find-tag' to keep the current functionality intact).
`find-definition' also keeps track of the tag ring, so this would move
`find-tag-marker-ring' and related functionality out of etags.el, too.

M-* is the standard opposite command for this, so that would be
extracted as well. SLIME and a few other modes re-define M-, to be the
opposite for M-. instead for easier navigation. How do you feel about
swapping the definition of M-, and M-* in etags.el?

C-M-. is currently bound to find-tag-regexp. There is currently no
standard functionality in Emacs to find the callers of a symbol at
point, which might be nice to put on C-M-. if it is defined at some
point for symmetry reasons. If so, find-tag-regexp needs a new key
binding. I'm unsure how useful find-tag-regexp is to begin with as I do
not use tags tables, so I kept it out of the proposal below, even
though it is certainly related.

Other commands affected by this would be C-x 4 . and C-x 5 .


So the new module find-definition.el would define:


Variable `find-definition-function'

A function to be called to find definitions for the symbol at point. It
is called with no arguments, and should return a marker for the
location of the definition.


M-. find-definition

Call `find-definition-function' and go to the marker returned, if any.
Store the old location in `find-definition-marker-ring'.


M-, or M-* find-definition-pop-mark

Go to the last location in `find-definition-marker-ring'


C-x 4 . and C-x 5 .

Same as `find-definition', except the marker returned by
`find-definition-function' is displayed in another window or frame
instead.


Changes to etags.el:

Remove the aforementioned key bindings.

Provide a new function, `etags-find-definition', which calls
`find-tag-noselect'. This will also check for the current prefix
argument to mimic the old behavior for NEXT-P.


Changes to elisp-mode.el:

Provide a value for `find-definition-function' which finds the
definition of the symbol at point.


Comments?

Regards,
Jorgen

Re: Generalizing find-definition

[Stefan Monnier]

> M-. is bound to a new command `find-definition', which primarily calls
> the value of a new variable `find-definition-function' (by default a
> wrapper around `find-tag' to keep the current functionality intact).
> `find-definition' also keeps track of the tag ring, so this would move
> `find-tag-marker-ring' and related functionality out of etags.el, too.

Yes, this sounds great.

> M-* is the standard opposite command for this, so that would be
> extracted as well.  SLIME and a few other modes re-define M-, to be the
> opposite for M-. instead for easier navigation.  How do you feel about
> swapping the definition of M-, and M-* in etags.el?

That's incompatible with the current M-, binding.
What would then be the equivalent of the current M-, ?

> C-M-. is currently bound to find-tag-regexp.  There is currently no
> standard functionality in Emacs to find the callers of a symbol at
> point, which might be nice to put on C-M-. if it is defined at some
> point for symmetry reasons.

M-. RET  does "find the callers of a symbol at point", AFAICT.

> Comments?

I'm all for it,


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

On Sun, 02 Nov 2014 10:34:28 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > M-* is the standard opposite command for this, so that would be
> > extracted as well.  SLIME and a few other modes re-define M-, to be
> > the opposite for M-. instead for easier navigation.  How do you
> > feel about swapping the definition of M-, and M-* in etags.el?
> 
> That's incompatible with the current M-, binding.
> What would then be the equivalent of the current M-, ?

The idea would be to simply swap M-, and M-*, so M-* would then be
`tags-loop-continue'. As I do not use tags, I do not know how often
that command is used and whether M-* is too inconvenient for this,
though.

> > C-M-. is currently bound to find-tag-regexp.  There is currently no
> > standard functionality in Emacs to find the callers of a symbol at
> > point, which might be nice to put on C-M-. if it is defined at some
> > point for symmetry reasons.
> 
> M-. RET  does "find the callers of a symbol at point", AFAICT.

I must be missing something - how does this work? M-. should jump
to the definition of the symbol at point, then RET should just enter a
newline? And if M-. prompts for a tag, RET will just accept the default?

Apparently, SLIME uses M-_ and M-? for edit-uses (to accomodate
differing keyboard layouts), so that might be a better choice anyhow.

> > Comments?
> 
> I'm all for it,

Thank you. I'll wait for more feedback and will work on this after the
official git transition.

Regards,
Jorgen

Re: Generalizing find-definition

[Helmut Eller]

On Sun, Nov 02 2014, Jorgen Schaefer wrote:

> On Sun, 02 Nov 2014 10:34:28 -0500
> Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
>> > M-* is the standard opposite command for this, so that would be
>> > extracted as well.  SLIME and a few other modes re-define M-, to be
>> > the opposite for M-. instead for easier navigation.  How do you
>> > feel about swapping the definition of M-, and M-* in etags.el?
>> 
>> That's incompatible with the current M-, binding.
>> What would then be the equivalent of the current M-, ?
>
> The idea would be to simply swap M-, and M-*, so M-* would then be
> `tags-loop-continue'. As I do not use tags, I do not know how often
> that command is used and whether M-* is too inconvenient for this,
> though.

In SLIME, M-, and M-* do the same thing.

Using M-, for pop-tag-mark is IMO very desirable, because on most
keyboards . and , are on adjacent keys and it's easy, almost pleasant,
to jump to a definition and back -- at least in the case when there's a
unique match.

We keep the binding for M-* around mostly because some users have stored
it in their muscle memory.  But it seems to me that M-* is quite hard to
press (on US keyboards) and if it weren't for tradition, we wouldn't
bind it at all.  SLIME will definitely keep the M-./M-, pair, well
knowing that it's incompatible with the binding for tags-loop-continue.

In my experience, tags-loop-continue is rather hard to use and I have
long argued to get rid of it and replace it with a better UI.
E.g. tags-loop-continue must be pressed multiple times just to find out
at the end that none of the offered candidates was relevant.  SLIME does
it differently: the list of all candidates is displayed in a separate
buffer with one candidate per line; a bit like the results of a search
engine like Google.  The user must then move the cursor to the
interesting line and press RET to actually jump to the definition.  If
there's only a single candidate, then there's no need to display the
list and we can jump to the candidate right away.  SLIME has no analog
to tags-loop-continue (and no key binding for it) because it's not
needed; at least nobody ever asked for such a command.

For SLIME, we would happily use the "standard" find-definition
infrastructure if it comes with a good UI.  The tags-loop-continue based
design is not acceptable for us.

Helmut

Re: Generalizing find-definition

[Jorgen Schaefer]

On Sun, 02 Nov 2014 19:14:55 +0100
Helmut Eller <eller.helmut@gmail.com> wrote:

> For SLIME, we would happily use the "standard" find-definition
> infrastructure if it comes with a good UI.

Thank you for the feedback.

Could you elaborate a bit on what you'd like to see for "a good UI"? My
idea so far was to simply have modes provide a function that returns a
marker or nil, while the function can check for interactive calls if it
wants to, and use that in the usual go to location in same
window/other window/other frame and go back. But I'm very open to ideas
to make this better.

Regards,
Jorgen

Re: Generalizing find-definition

[Helmut Eller]

On Sun, Nov 02 2014, Jorgen Schaefer wrote:

> Could you elaborate a bit on what you'd like to see for "a good UI"? My
> idea so far was to simply have modes provide a function that returns a
> marker or nil, while the function can check for interactive calls if it
> wants to, and use that in the usual go to location in same
> window/other window/other frame and go back. But I'm very open to ideas
> to make this better.

Let me define a bit of terminology that I use in SLIME: M-. parses the
"symbol" at point and then asks the external process to search a list of
"candidates" that match the symbol.  A candidate is a "location" with a
short description.  A location is a file:line:column triple (in the
simplest case).

The primary complication for a "good UI" is the case when there is more
than one candidate.

etags.el solves this complication by introducing the tags-loop-continue
command to cycle through the list of candidates (I think etags.el does
not compute an explicit list of candidates, but instead keeps a marker
in the TAGS buffer and tags-loop-continue will quite literally continue
the search starting from that marker).  I don't like this kind of UI.
It would be better to display the list of candidates instead of keeping
it hidden.

Displaying the list of candidates in an auxiliary buffer, make it easy to
select an interesting candidate and finally close the buffer is the job
of the UI.  Doing this well is not so easy.  I spent quite some time to
make it good in SLIME.  It's still not perfect, but IMO way better than
what etags.el offers.

Parsing the "symbol" at point for M-. will be language dependent and
some language may need to determine a lot of context (current
module/class/function etc.).  In SLIME, it's relatively easy as we only
need to determine the current package and then the symbol like in Elisp.
I think a "good UI" should, by default, determine the symbol around
point without asking each time (as find-tag does).

BTW: M-x rgrep needs to solve similar UI issues as it also needs to deal
with a "list of candidates".  Maybe there's an opportunity to
share/unify some things.

Helmut

Re: Generalizing find-definition

[Jorgen Schaefer]

On Sun, 02 Nov 2014 20:51:40 +0100
Helmut Eller <eller.helmut@gmail.com> wrote:

> On Sun, Nov 02 2014, Jorgen Schaefer wrote:
> 
> > Could you elaborate a bit on what you'd like to see for "a good
> > UI"? My idea so far was to simply have modes provide a function
> > that returns a marker or nil, while the function can check for
> > interactive calls if it wants to, and use that in the usual go to
> > location in same window/other window/other frame and go back. But
> > I'm very open to ideas to make this better.
> 
> Let me define a bit of terminology that I use in SLIME:
> [...]

Thank you for the elaborate explanation!

> The primary complication for a "good UI" is the case when there is
> more than one candidate.

Yes; I deliberately omitted this for my initial proposal, as this is
definitely not simple, and I suspect no matter how it is done, it will
leave *someone* unhappy.

> BTW: M-x rgrep needs to solve similar UI issues as it also needs to
> deal with a "list of candidates".  Maybe there's an opportunity to
> share/unify some things.

I think it would be possible to optionally return a list of markers
from `find-definition-function', which would then be displayed in a
buffer similar to rgrep/compile.

Thank you for the idea. I like that.

Regards,
Jorgen

Re: Generalizing find-definition

[Stefan Monnier]

> The idea would be to simply swap M-, and M-*, so M-* would then be
> `tags-loop-continue'.

I'm not sure what would be the benefit of such a swap.

This said, one of the benefits of the generic functionality you're
suggesting is that users can easily rebind those commands and know that
this will affect consistently all packages providing the functionality.

>> M-. RET  does "find the callers of a symbol at point", AFAICT.

Duh, sorry for being dense, of course it doesn't.  etags does not offer
the needed info, so the etags.el UI doesn't have such a thing.

Helmut Eller <eller.helmut@gmail.com> writes:
> bind it at all.  SLIME will definitely keep the M-./M-, pair, well
> knowing that it's incompatible with the binding for tags-loop-continue.

When SLIME uses the new functionality it should mess with its key
bindings at all, so the user can more easily make global changes to
those key bindings.

> In my experience, tags-loop-continue is rather hard to use and I have
> long argued to get rid of it and replace it with a better UI.
> E.g. tags-loop-continue must be pressed multiple times just to find out
> at the end that none of the offered candidates was relevant.  SLIME does
> it differently: the list of all candidates is displayed in a separate
> buffer with one candidate per line; a bit like the results of a search
> engine like Google.  The user must then move the cursor to the
> interesting line and press RET to actually jump to the definition.  If
> there's only a single candidate, then there's no need to display the
> list and we can jump to the candidate right away.  SLIME has no analog
> to tags-loop-continue (and no key binding for it) because it's not
> needed; at least nobody ever asked for such a command.

I agree that tags-loop-continue is not super convenient.
What key-binding does SLIME use to get this list buffer (which would
most naturally be implemented as a kind of grep/compilation-mode buffer)?
Would C-u M-. be usable for that?

> ada-mode also has "show all references of name at point", which
> currently pops up a buffer in compilation-mode, showing
> compilation-style locations of all uses.  That's similar functionality to
> find-tag/tags-loop-continue, and similar UI to SLIME.

Sounds like we agree.

So on the UI side we mostly have:
- "jump to *the* definition of thing at point".
- "list definitions of thing at point".
- "list uses of thing at point".
- "return to buffer/position before the last jump".

And on the backend side we have:
- identifier-at-point-function
- find-definition-function (with an argument to decide whether we want
  a whole list or just the best/first candidate).
- find-uses-function

Etags.el currently offers some additional functionality:
- jump to definition of any identifier, with TAB-completion.
- jump to file (among those listed in the TAGS file), tho currently
  without TAB-completion.
AFAICT, the first could be implemented just in the generic code, except
for TAB-completion.
And the second one is also provided by things like projectile (among
many other thingies), tho it seems that none of the bundled Emacs
packages provide a good solution for that common need.


        Stefan

Re: Generalizing find-definition

[Helmut Eller]

On Sun, Nov 02 2014, Stefan Monnier wrote:

>> In my experience, tags-loop-continue is rather hard to use and I have
>> long argued to get rid of it and replace it with a better UI.
>> E.g. tags-loop-continue must be pressed multiple times just to find out
>> at the end that none of the offered candidates was relevant.  SLIME does
>> it differently: the list of all candidates is displayed in a separate
>> buffer with one candidate per line; a bit like the results of a search
>> engine like Google.  The user must then move the cursor to the
>> interesting line and press RET to actually jump to the definition.  If
>> there's only a single candidate, then there's no need to display the
>> list and we can jump to the candidate right away.  SLIME has no analog
>> to tags-loop-continue (and no key binding for it) because it's not
>> needed; at least nobody ever asked for such a command.
>
> I agree that tags-loop-continue is not super convenient.
> What key-binding does SLIME use to get this list buffer (which would
> most naturally be implemented as a kind of grep/compilation-mode buffer)?
> Would C-u M-. be usable for that?

M-. brings up the list (if there are multiple candidates; if there's a
single candidate then no list is displayed).  We use C-u M-. to read the
symbol from the minibuffer (as opposed to parsing it from context around
point).

We also have a group of bindings for similar but not so important
commands:

 C-c C-w c  -- who calls (lists callers)
 C-c C-w w  -- calls who (lists functions called by the current function)
 C-c C-w r  -- who references (for global variables)
 C-c C-w s  -- who sets (for global variables)
 C-c C-w b  -- who binds (for dynamic variables)
 C-c C-w s  -- who specializes (lists methods specialized for a specific class)
 C-c <      -- list callers
 C-c >      -- list callees

The C-c < and C-c C-w c do conceptually the same but one is implemented
with an index (like tags) and the other by scanning and inspecting
function objects on the heap.  Different commands for historical
reasons.

Helmut

Re: Generalizing find-definition

[Jorgen Schaefer]

On Sun, 02 Nov 2014 21:22:03 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > The idea would be to simply swap M-, and M-*, so M-* would then be
> > `tags-loop-continue'.
> 
> I'm not sure what would be the benefit of such a swap.

The only benefit would be that M-, is now "go back after M-.", which is
what a lot of the packages that redefine M-. do to the key. Hence the
suggestion to swap the two keys, to unify that apparently rather
widespread use of M-,. If Emacs would prefer to keep M-, as is, that is
fine with me, too.

> What key-binding does SLIME use to get this list buffer (which would
> most naturally be implemented as a kind of grep/compilation-mode
> buffer)? Would C-u M-. be usable for that?

All of SLIME, slime-nav, Geiser and Cider use C-u M-. to prompt the
user for a symbol name to go to instead of using the symbol at point,
so that might be a useful generalized function if we do not need the
prefix for the "next" feature of etags.

> > ada-mode also has "show all references of name at point", which
> > currently pops up a buffer in compilation-mode, showing
> > compilation-style locations of all uses.  That's similar
> > functionality to find-tag/tags-loop-continue, and similar UI to
> > SLIME.
> 
> Sounds like we agree.
> 
> So on the UI side we mostly have:
> - "jump to *the* definition of thing at point".
> - "list definitions of thing at point".

Agreed. So far, I would say this is the same key binding (M-.)
which then either jumps to the definition if there is one, or lists them
if there is more than one.

> - "list uses of thing at point".

What key binding would you suggest for that? SLIME apparently uses M-_
and M-? (i.e. the key right of . in different keyboard layouts).

> - "return to buffer/position before the last jump".

Agreed.

> And on the backend side we have:
> - identifier-at-point-function

Why do we need this? It does not make much sense for this functionality
in languages with combined identifiers. For example, in Python,
"foo.bar()" can be pretty much anything, it depends heavily on the
context. Libraries for introspection expect a position in a file/buffer,
not an identifier, to find the definitions.

> - find-definition-function (with an argument to decide whether we want
>   a whole list or just the best/first candidate).

I am not sure if it is generally easily possible to sort the list of
candidates like that. I guess tags often has the problem of knowing
more than one possible candidate, so it would be preferable for tags to
default to a single destination?

> Etags.el currently offers some additional functionality:
> - jump to definition of any identifier, with TAB-completion.

C-u M-. could call a separate function when set, which would be
provided by the mode author to prompt for an identifier (with tab
completion if possible) and goes to the definition of that symbol.

Regards,
Jorgen

Re: Generalizing find-definition

[Stephen Leake]

Jorgen Schaefer <forcer@forcix.cx> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
>> And on the backend side we have:
>> - identifier-at-point-function
>
> Why do we need this? It does not make much sense for this functionality
> in languages with combined identifiers. For example, in Python,
> "foo.bar()" can be pretty much anything, it depends heavily on the
> context. Libraries for introspection expect a position in a file/buffer,
> not an identifier, to find the definitions.

ada-mode has ada-identifier-at-point; it returns a string suitable for
the external cross-reference engine.

On the other hand, 'ada-goto-declaration' (which would be used for
'find-definition-function') calls ada-identifier-at-point (and similarly
for other cases), so I don't think this needs to be split out as an API
function.

>> - find-definition-function (with an argument to decide whether we want
>>   a whole list or just the best/first candidate).
>
> I am not sure if it is generally easily possible to sort the list of
> candidates like that. I guess tags often has the problem of knowing
> more than one possible candidate, so it would be preferable for tags to
> default to a single destination?

I don't ever want the back-end to guess for me without giving me the
option to override; if there's a list, I want to see it, either as an
icomplete style completion list, or in a buffer.

If the backend can put some order on the list (last used is usually
good), that is fine.

>> Etags.el currently offers some additional functionality:
>> - jump to definition of any identifier, with TAB-completion.
>
> C-u M-. could call a separate function when set, which would be
> provided by the mode author to prompt for an identifier (with tab
> completion if possible) and goes to the definition of that symbol.

It might also be useful to have find-definition-function take an
optional 'identifier' argument. Then other code that finds identifiers
could use it.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> The only benefit would be that M-, is now "go back after M-.", which is
> what a lot of the packages that redefine M-. do to the key.  Hence the
> suggestion to swap the two keys, to unify that apparently rather
> widespread use of M-,.  If Emacs would prefer to keep M-, as is, that is
> fine with me, too.

I see, so it's compatibility with other packages vs. compatibility with
etags.el.  Luckily, I think there is no hurry to make this choice.
So we could poll the users.

>> So on the UI side we mostly have:
>> - "jump to *the* definition of thing at point".
>> - "list definitions of thing at point".
> Agreed. So far, I would say this is the same key binding (M-.)
> which then either jumps to the definition if there is one, or lists them
> if there is more than one.

IIUC the first is only used when there's only one definition.  I guess
it's OK, and it reduces the pressure on key-bindings.

>> - "list uses of thing at point".
> What key binding would you suggest for that? SLIME apparently uses M-_
> and M-? (i.e. the key right of . in different keyboard layouts).

AFAICT there're both unbound currently, so either is fine.  Of course,
we could also re-use M-. for that (see below).

>> And on the backend side we have:
>> - identifier-at-point-function
> Why do we need this?

I thought we already agreed that we need a language-aware way to
determine what is the identifier at point (taking into account the
current namespace/package/etc..).

> It does not make much sense for this functionality in languages with
> combined identifiers.  For example, in Python, "foo.bar()" can be
> pretty much anything, it depends heavily on the context.  Libraries for
> introspection expect a position in a file/buffer, not an identifier,
> to find the definitions.

But if we also want the user to type the identifier via `C-u M-.', then
we need find-definition-function to be able to deal with a string rather
than a buffer position.  So let's simply say that
find-definition-function will be called either with a string or with
a buffer position.  If the function prefers only working from strings,
it can call identifier-at-point-function to turn the buffer position
into a string.

>> - find-definition-function (with an argument to decide whether we want
>> a whole list or just the best/first candidate).
> I am not sure if it is generally easily possible to sort the list of
> candidates like that.  I guess tags often has the problem of knowing
> more than one possible candidate, so it would be preferable for tags to
> default to a single destination?

In etags.el, it doesn't take noticeably more time to get the full list
rather than just the first match, so I guess it's OK if we just always
return the full list.

>> Etags.el currently offers some additional functionality:
>> - jump to definition of any identifier, with TAB-completion.
> C-u M-. could call a separate function when set, which would be
> provided by the mode author to prompt for an identifier (with tab
> completion if possible) and goes to the definition of that symbol.

Right.  I guess the prompting can be part of the generic code, and the
completion-table can be provided by an appropriate foo-function set by
the backend providing find-definition-function.

[ Side note: I expect that identifier-at-point-function would pretty
  much always be provided by the major mode, but I expect that
  find-definition-function (as well as the TAB-completion function) will
  sometimes be provided by the major mode, and sometimes by
  a mode-agnostic package (like etags.el).  ]

Helmut Eller <eller.helmut@gmail.com> writes:
> We also have a group of bindings for similar but not so important
> commands:
>
>  C-c C-w c  -- who calls (lists callers)
>  C-c C-w w  -- calls who (lists functions called by the current function)
>  C-c C-w r  -- who references (for global variables)
>  C-c C-w s  -- who sets (for global variables)
>  C-c C-w b  -- who binds (for dynamic variables)
>  C-c C-w s  -- who specializes (lists methods specialized for a specific class)
>  C-c <      -- list callers
>  C-c >      -- list callees

Right, this brings up two issues:
- distinguish different kinds of uses: definitions, accesses,
  assignments, refinements (via advice-add, defmethod, ...), let-binds, ...
- distinguish different categories of "identifiers" (e.g. classes,
  variables, functions, types, methods, ...).

So far the proposal is to forget about the second distinction, tho if we
pass buffer-positions to find-definition-function,
find-definition-function can make those distinctions on its own.
We could additionally have some kind of convention for disambiguation
when using identifier strings, e.g. type "foobar variable" in the
minibuffer prompt to state explicitly that you're interested in the
foobar variable rather than the foobar type.

For the first distinction, at the UI level we could do:
- Specify the kind of use by the key-binding (M-. to find
  definitions, M-? to find uses).  This doesn't work too well if there
  are many more refinements.
- Specify the kind of use via a minibuffer prompt.
  E.g. C-u M-. would first prompt for an identifier (defaulting to the
  one at point), and then prompt for the kind of use to look for.
- Specify the use by filtering in the "found matches" buffer.
  For "definition", this kind of sucks since in many cases there'd be
  only one definition, but you'd first have to go through the complete
  list displayed in a buffer, and then use some key-binding to filter
  the sole definition out of that.
- A mix of the above.  E.g. M-. searches only for one particular subset
  of uses (e.g. definitions and refinements), while M-? search for the
  complementary subset (or for all possible uses, including the ones
  already provided under M-.), and after M-? you can filter the results
  in the buffer.

At the backend level, in order to accommodate those different possible
UIs, the find-definition-function would seem to need to receive 2 args:
one specifying the identifier, and the other specifying the kind of uses
and the elements of the returned list should not just be positions but
pairs of "position and use-kind".


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

On Mon, 03 Nov 2014 09:30:56 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> >> And on the backend side we have:
> >> - identifier-at-point-function
> > Why do we need this?
> 
> I thought we already agreed that we need a language-aware way to
> determine what is the identifier at point (taking into account the
> current namespace/package/etc..).

For Python, there is nothing suitable for "identifier at point" that I
can return that would help find the definition, except "the whole
contents of the buffer, the file name, and the position of point within
the buffer".

To get the definition of something, I pass that - the full text, the
file name, and the position - to a Python library and get back a list of
locations.

There is no intermediate "get the identifier at point" step, because
that concept does not make sense here.

> >> Etags.el currently offers some additional functionality:
> >> - jump to definition of any identifier, with TAB-completion.
> > C-u M-. could call a separate function when set, which would be
> > provided by the mode author to prompt for an identifier (with tab
> > completion if possible) and goes to the definition of that symbol.
> 
> Right.  I guess the prompting can be part of the generic code, and the
> completion-table can be provided by an appropriate foo-function set by
> the backend providing find-definition-function.

For Python, I do provide a tab-completable list of identifiers
(currently only for pydoc, but the code should be reusable for this),
but it is not a simple list of identifiers, but a tree. The tab
completion starts with top-level modules (like "json"), but when you
add a ".", it will continue to complete attributes of the module or
class etc.. Providing the full list of possible completions right
away would take a very long time. This kind of completion is tricky to
generalize, so if we want the user to be able to type a symbol, we
should allow the backend to override this. We can provide a simple
default for the trivial case, though.

So as I see it, `find-definition' would usually call the value of
`find-definition-function' with no arguments. If that function returns
an empty list, or if `find-definition' was called with a prefix
argument, it would use a function from the backend that prompts for
a symbol and pass the result of that to `find-definition-function'.

Alternatively, the backend simply provides one function that prompts
the user for an identifier and returns the list of uses instead of the
dance above.

> [ Side note: I expect that identifier-at-point-function would pretty
>   much always be provided by the major mode, but I expect that
>   find-definition-function (as well as the TAB-completion function)
>   will sometimes be provided by the major mode, and sometimes by
>   a mode-agnostic package (like etags.el).  ]

Or even a minor mode that provide extra functionality for a major mode -
Elpy is a minor mode which is active in Python mode buffers.

> For the first distinction, at the UI level we could do:
> - Specify the kind of use by the key-binding (M-. to find
>   definitions, M-? to find uses).  This doesn't work too well if there
>   are many more refinements.
> - Specify the kind of use via a minibuffer prompt.
>   E.g. C-u M-. would first prompt for an identifier (defaulting to the
>   one at point), and then prompt for the kind of use to look for.
> - Specify the use by filtering in the "found matches" buffer.
>   For "definition", this kind of sucks since in many cases there'd be
>   only one definition, but you'd first have to go through the complete
>   list displayed in a buffer, and then use some key-binding to filter
>   the sole definition out of that.
> - A mix of the above.  E.g. M-. searches only for one particular
> subset of uses (e.g. definitions and refinements), while M-? search
> for the complementary subset (or for all possible uses, including the
> ones already provided under M-.), and after M-? you can filter the
> results in the buffer.

This is getting extremely complex and I do not see how this would
benefit the user much. There are definitions of an identifier, and uses
of an identifier.

Typical IDEs make a distinction between these, too. For example,
Eclipse provides "go to definition or declaration" on F3 or <C-left> on
a symbol.

The *uses* of an identifier are quite tricky. IDEs provide various
tools for that, including various searches, call graphs, highlighting
occurences, etc.

There are usually very few definitions, and M-. should just go there.
The "find uses of the identifier at point" functionality seems quite
distinct from a user's point of view, so should be distinct in the user
interface, too.

Regards,
Jorgen

Re: Generalizing find-definition

[Stefan Monnier]

> For Python, there is nothing suitable for "identifier at point" that I

Yet, you say that you offer a tab-completable selection of identifiers.
So obviously, there is a suitable "identifier at point".

Do you mean by the above not that the concept is meaningless but that
it's difficult to write a function that reliably returns the appropriate
identifier at point?

If so, I can definitely agree that for some modes it can be difficult,
and it should not be necessary to write such a function if the backend
can use a buffer-position instead (and delegate the hard work to some
external tool).

But having such an identifier-at-point-function can be useful for all
kinds of things (typically default for minibuffer inputs of various
commands), so it makes a lot of sense to standardize it.

> For Python, I do provide a tab-completable list of identifiers
> (currently only for pydoc, but the code should be reusable for this),
> but it is not a simple list of identifiers, but a tree. The tab
> completion starts with top-level modules (like "json"), but when you
> add a ".", it will continue to complete attributes of the module or
> class etc.. Providing the full list of possible completions right
> away would take a very long time.

Of course.  But such completion is nothing new.  File names are
completed this way.  Bzr revision names are completed this way as well.
No need for a special completion command, you can do that just fine with
just a completion-table (and then partial-completion style will let you
complete "js.fo" to anything that matches "js*.fo*").

Note that such completion tables are clearly not lists of strings, but
they're functions (actually, they're objects represented as functions,
for lack of an object system).

> This is getting extremely complex and I do not see how this would
> benefit the user much.

As long as the "set of use-kinds" and the "set of identifier categories"
is determined by the backend itself, I don't think it increases
complexity of the backends or the backend API very much.

And there are clearly example of existing systems which do provide such
refinements (e.g. find-variable vs find-function on Elisp, and the C-c
C-w s versus C-c C-w b versus C-c C-w r in SLIME), so it makes sense to
include it in the design of the backend API.

> There are definitions of an identifier, and uses of an identifier.

Yes, clearly this is the main distinction and we should focus on this
functionality w.r.t designing the UI.


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

On Mon, 03 Nov 2014 15:09:40 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > For Python, there is nothing suitable for "identifier at point"
> > that I
> 
> Yet, you say that you offer a tab-completable selection of
> identifiers. So obviously, there is a suitable "identifier at point".

The tab completion I talked about is for global modules and their
contents and has no access to locally-available identifiers. It is a
feature completely unrelated to finding the definition of the
identifier at point.

> Do you mean by the above not that the concept is meaningless but that
> it's difficult to write a function that reliably returns the
> appropriate identifier at point?

I suspect we mean the same thing, but just to clarify.

Consider a piece of code like this:

    from foo import Foo

    bar = Foo()
    bar.baz_|_

M-. should go to the definition of the attribute baz of the class Foo.
But the obvious "identifier at point" is "bar.baz", which by itself
does not have any relationship to said method without the assignment
"bar = Foo()", which by itself is also not meaningful without the
import statement. The only single string that reliably would allow to
find the correct definition would be "foo.Foo.baz", but I do not think
that anyone would consider that to be "the identifier at point" here.

So what I am saying is that the only representation for "the identifier
at point" that allows finding the definition of said identifier is the
triple (file name, full buffer contents, position within the buffer
contents). There is no simpler "identifier at point".

> If so, I can definitely agree that for some modes it can be difficult,
> and it should not be necessary to write such a function if the backend
> can use a buffer-position instead (and delegate the hard work to some
> external tool).
>
> But having such an identifier-at-point-function can be useful for all
> kinds of things (typically default for minibuffer inputs of various
> commands), so it makes a lot of sense to standardize it.

Emacs can and probably should standardize it, but unless it is useful
for the interface of finding the definition of the thing at point, it
would seem like a good idea not to do it in the library being discussed
here.

> Note that such completion tables are clearly not lists of strings, but
> they're functions (actually, they're objects represented as functions,
> for lack of an object system).

What is the advantage of having the backend define a function that
returns a completion table as opposed to a function that prompts the
user for a symbol? The saved code seems minimal, and it means the
backend can not set the prompt, for example.

Regards,
Jorgen

Re: Generalizing find-definition

[Stefan Monnier]

> The only single string that reliably would allow to find the correct
> definition would be "foo.Foo.baz", but I do not think that anyone
> would consider that to be "the identifier at point" here.

I would.

>> Note that such completion tables are clearly not lists of strings, but
>> they're functions (actually, they're objects represented as functions,
>> for lack of an object system).
> What is the advantage of having the backend define a function that
> returns a completion table as opposed to a function that prompts the
> user for a symbol? The saved code seems minimal, and it means the
> backend can not set the prompt, for example.

It means the UI can use it for various kinds of completion (e.g. for
completion-at-point-function).


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > from foo import Foo

> > bar = Foo()
> > bar.baz_|_

>> The only single string that reliably would allow to find the correct
>> definition would be "foo.Foo.baz", but I do not think that anyone
>> would consider that to be "the identifier at point" here.
>
> I would.

I would also, but I would not expect a generic Emacs function to return
that. It certainly requires a backend function.

I'm still not clear what 'identifier-at-point' is good for outside "goto
definition", "show uses", and completion. Since the meaning is highly
backend specific, I don't see what other parts of Emacs might use it.

You say there are other uses - can you list some?

In ada-mode, 'ada-identifier-at-point' is used in (ada-mode doesn't
provide completion yet):

ada-goto-declaration

    aka find-definition-function
    
ada-show-references

    aka find-uses-function
    
ada-show-declaration-parents

    If identifier-at-point is a type, show the definition of the types
    it inherits from.
    
ada-show-overriding

    If identifier-at-point is a function, show definitions
    of functions that override it.
    
ada-show-overridden

    If identifier-at-point is a function, show definitions
    of the function that it overrides.

The last three are additional candidates for the API.

>>> Note that such completion tables are clearly not lists of strings, but
>>> they're functions (actually, they're objects represented as functions,
>>> for lack of an object system).
>> What is the advantage of having the backend define a function that
>> returns a completion table as opposed to a function that prompts the
>> user for a symbol? The saved code seems minimal, and it means the
>> backend can not set the prompt, for example.
>
> It means the UI can use it for various kinds of completion (e.g. for
> completion-at-point-function).

+1

ada-mode would provide a tree as well.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> I would also, but I would not expect a generic Emacs function to return
> that.  It certainly requires a backend function.

Agreed.

> I'm still not clear what 'identifier-at-point' is good for outside "goto
> definition", "show uses", and completion. Since the meaning is highly
> backend specific, I don't see what other parts of Emacs might use it.

I just know that it's used commonly.  Maybe it's always used by
backend-specific code, so it could stay completely in each backend, but
I don't see any reason why we couldn't standardize the name of that
backend function, in case some non-backend specific code wants to
use it.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I would also, but I would not expect a generic Emacs function to return
>> that.  It certainly requires a backend function.
>
> Agreed.
>
>> I'm still not clear what 'identifier-at-point' is good for outside "goto
>> definition", "show uses", and completion. Since the meaning is highly
>> backend specific, I don't see what other parts of Emacs might use it.
>
> I just know that it's used commonly.  Maybe it's always used by
> backend-specific code, so it could stay completely in each backend, but
> I don't see any reason why we couldn't standardize the name of that
> backend function, in case some non-backend specific code wants to
> use it.

The reason we can't standardize it is that "identifier-at-point" appears
to have different meanings/different results in different situations.

For "goto definition" and "show uses", ada-mode needs the simple word at
point (no dots; 'baz' in the example), together with the file name,
line, and column. "ada-identifier-at-point" only returns the simple
word, not all the information needed by "goto definition". Python needs
the file name, buffer text, line and column; it does not have an
implementation of "identifier-at-point", apparently.

For "completion", Python also apparently needs the same information as
for "goto definition". Ada mode will probably need the same info as for
"goto definition" as well.

None of these make sense for something called "identifier at point".
"xref-info-at-point" might make sense.

So without a list of other uses, it's hard to know how to define
"identifier-at-point".

I grepped thru ada-*.el for "-at-point", and found:

ada-case-adjust-at-point

    Adjust case of the simple word before point according to the casing
    mode (typically capitalize first letter). Does _not_ call
    ada-identifier-at-point. "word" is defined by the syntax classes
    word and symbol. 

ada-identifier-at-point

    discussed above

(thing-at-point 'symbol)

    Used to retrieve the simple word for the external xref engine, when
    in a C++ buffer; same meaning as ada-identifier-at-point. The
    external xref engine used by ada-mode is multi-lingual (see
    gpr-query in the ada-mode sources).

Note that thing-at-point does _not_ have an option 'identifier. Also
note that the meaning of 'symbol is not given anywhere; the docstring
for thing-at-point says "see thingatpt.el", but I don't see a definition
there. So apparently the definition is "whatever (thing-at-point
'symbol) returns", which is not very helpful.

The default implementation of (thing-at-point 'symbol) relies on the
word and symbol syntax classes, which is why it works for C++ mode. That
won't work for Ada mode, because "*" is an identifier for a function
that overrides *. (Hmm, I'm not sure 'symbol works for overloaded C++
operators, now that I think about it).

Note that the default implementation can be overridden by setting symbol
properties. So we could require a backend to do that to return the above
info, so "goto definition" could start by calling (thing-at-point
'symbol). But if we did take that approach, I'd argue it should be
(thing-at-point 'xref) instead.

So I'm still puzzled as to what other uses "identifier-at-point" might
have. If they are all either satisfied by thing-at-point or are strictly
back-end-specific, then we don't need anything new.

I grep'd thru emacs/lisp/* for "identifier-at-point"; there were _no_
occurrences.

I started grep'ing thru emacs/lisp/* for "-at-point". So far I've found
(ignoring things that don't return strings):

allout-parse-symbol-at-point - appears to be specific to allout

highlight-symbol-at-point - uses find-tag-default-as-symbol-regexp,
    which defaults to word and symbol syntax classes; could be
    (thing-at-point 'symbol).

thing-at-point - used by calc, cmuscheme, others

number-at-point - used by calc, defined in thingatpt.el, uses
    (thing-at-point 'sexp)

completion-at-point-functions - bound by functions in semantic, other modes

semantic-completion-at-point-function - defaults to
    semantic-ia-complete-symbol, which uses parse results. I'll call
    that backend-specific.

variable-at-point - used by cus-edit.el, uses word and symbol syntax
    classes, then does some more checks on whether it is a lisp symbol.
    could use (thing-at-point 'symbol)

... and I got tired. So far it's either backend or (thing-at-point
'symbol). There are about 1900 matches; that's a lot of information to
sort thru. 

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> For "goto definition" and "show uses", ada-mode needs the simple word at
> point (no dots; 'baz' in the example), together with the file name,
> line, and column. "ada-identifier-at-point" only returns the simple
> word, not all the information needed by "goto definition". Python needs
> the file name, buffer text, line and column; it does not have an
> implementation of "identifier-at-point", apparently.

That's OK.  We already agreed that find-definition-function should
accept a buffer-position as argument (from which it can extract any
identifier it likes, if that's what it needs).

> For "completion", Python also apparently needs the same information as
> for "goto definition". Ada mode will probably need the same info as for
> "goto definition" as well.

Completion doesn't need the identifier at point, AFAICT.


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The only single string that reliably would allow to find the correct
>> definition would be "foo.Foo.baz", but I do not think that anyone
>> would consider that to be "the identifier at point" here.
>
> I would.

I think it should return "baz" here, if we want to go this way at all.
And then that value could be passed to a "search-definition-function",
which will perform a full non-precise scan.

It apparent that we won't be able to do precise searches for identifiers
in most cases, so having a separate variable and function for this seems
to be the way to go, considering that the implementations are likely to
be fairly different anyway.

Re: Generalizing find-definition

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> The only single string that reliably would allow to find the correct
>>> definition would be "foo.Foo.baz", but I do not think that anyone
>>> would consider that to be "the identifier at point" here.
>>
>> I would.
>
> I think it should return "baz" here, if we want to go this way at all.
> And then that value could be passed to a "search-definition-function",
> which will perform a full non-precise scan.

When would that be useful?

> It apparent that we won't be able to do precise searches for identifiers
> in most cases, 

It requires support from the compiler/backend, but that should always be
available (presumably you have the compiler if you are editing the
code). So why do you think we can't do precise searches in most cases?

Hmm, maybe you won't have the compiler installed if you are just reading
someone else's code. I'm not sure that's an important use case.

-- 
-- Stephe

Re: Generalizing find-definition

[Yuri Khan]

On Fri, Nov 7, 2014 at 1:40 AM, Stephen Leake
<stephen_leake@stephe-leake.org> wrote:

>> I think it should return "baz" here, if we want to go this way at all.
>> And then that value could be passed to a "search-definition-function",
>> which will perform a full non-precise scan.
>
> When would that be useful?

When function names are unique enough between interfaces and the
parser is not good enough to precisely identify which subset of
classes the current invocation may involve. The search-definition
engine may then say “oh screw it, I got confused, here are all
definitions of baz in all of your bazillion classes, figure it out
yourself, you’re human after all”.

>> It apparent that we won't be able to do precise searches for identifiers
>> in most cases,
>
> It requires support from the compiler/backend, but that should always be
> available (presumably you have the compiler if you are editing the
> code). So why do you think we can't do precise searches in most cases?

Some compilers don’t expose the necessary information in a
machine-readable form, or are not sufficiently fast to produce a
result within the interval the user is prepared to wait.

Re: Generalizing find-definition

[Dmitry Gutov]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

>> I think it should return "baz" here, if we want to go this way at all.
>> And then that value could be passed to a "search-definition-function",
>> which will perform a full non-precise scan.
>
> When would that be useful?

In general? In situations similar to whenever a user would call
`apropos' when working with ELisp. Sometimes you don't actually know
what you're looking for, so you try one or several keywords.

>> It apparent that we won't be able to do precise searches for identifiers
>> in most cases, 
>
> It requires support from the compiler/backend, but that should always be
> available (presumably you have the compiler if you are editing the
> code). So why do you think we can't do precise searches in most cases?

You're assuming a statically typed language. Arguably, people writing in
dynamic languages are at least a significant portion of Emacs users, if
not the majority.

And C/C++ users will also be at a disadvantage here, unless they use a
Clang-based tool, which we won't support in Emacs proper at least in the
near future. Or GCC developers deliver on that front.

> Hmm, maybe you won't have the compiler installed if you are just reading
> someone else's code. I'm not sure that's an important use case.

If we don't have a tool which can respond to a
search-definition-function request, this is not an interesting situation
anyway, even if a plausible one.

Re: Generalizing find-definition

[Stefan Monnier]

> What is the advantage of having the backend define a function that
> returns a completion table as opposed to a function that prompts the
> user for a symbol? The saved code seems minimal, and it means the
> backend can not set the prompt, for example.

Another reason is the general rule that the backend should not do
UI tasks and prompting is definitely a UI operation.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> What is the advantage of having the backend define a function that
>> returns a completion table as opposed to a function that prompts the
>> user for a symbol? The saved code seems minimal, and it means the
>> backend can not set the prompt, for example.
>
> Another reason is the general rule that the backend should not do
> UI tasks and prompting is definitely a UI operation.

+1

-- 
-- Stephe

Re: Generalizing find-definition

[Stephen J. Turnbull]

Jorgen Schaefer writes:

 > The only single string that reliably would allow to find the
 > correct definition would be "foo.Foo.baz", but I do not think that
 > anyone would consider that to be "the identifier at point" here.

Why not?  Emacs's practical definition of "identifier at point"
doesn't need to be the same as that of the programming language's
grammar.  I have no trouble taking both points of view, although
usually at different times.

 > So what I am saying is that the only representation for "the identifier
 > at point" that allows finding the definition of said identifier is the
 > triple (file name, full buffer contents, position within the buffer
 > contents). There is no simpler "identifier at point".

In some sense, you always need full buffer contents except for the
very simplest of programs.

Re: Generalizing find-definition

[Jorgen Schaefer]

On Tue, 04 Nov 2014 08:46:33 +0900
"Stephen J. Turnbull" <stephen@xemacs.org> wrote:

> Jorgen Schaefer writes:
> 
>  > The only single string that reliably would allow to find the
>  > correct definition would be "foo.Foo.baz", but I do not think that
>  > anyone would consider that to be "the identifier at point" here.
> 
> Why not?

The "foo.Foo.baz" in this example would be an error at that point in
the program, and not all things that are defined have such a fully
qualified name in the first place. This all seems a bit
counter-intuitive for something called "identifier at point".

> Emacs's practical definition of "identifier at point"
> doesn't need to be the same as that of the programming language's
> grammar.  I have no trouble taking both points of view, although
> usually at different times.

I have no problem with that, either - we can call it Frobble, Blubber
or Moof, too, if we like. We use words so others understand us, and
redefining them more or less randomly is not helping with communication.

We have gone full circle and instead of asking "can an 'identifier at
point' be used to find the definition of the thing at point", we have
defined "identifier at point" to mean "that string (or object?) which
can be used to find the definition of the thing at point". Now that is
fine, mind you, we can use language however we like, but we will cause
quite some confusion with that.

For example, some people might think "oh hey, I have a function that
returns an identifier at point, and I am supposed to be able to pass
that string to the other function which returns the definitions of that
identifier. Let's write a function which receives an identifier and
returns where it is used."

But the way we defined "identifier at point", that does not necessarily
allow that.

Confusion all around just because we went for re-defining a term that
is usually used in other ways.

> In some sense, you always need full buffer contents except for the
> very simplest of programs.

Exactly. Which is why I'm saying that I do not see the need or use for
an indirection via a "get identifier at point" function for the
proposed library. Which is pretty much all I was trying to say up to the
semantics discussion on the number of ways we can define what
"identifier at point" could possibly mean.

So let's drop that idea and the semantics discussion.

Regards,
Jorgen

Re: Generalizing find-definition

[Yuri Khan]

On Tue, Nov 4, 2014 at 2:55 AM, Jorgen Schaefer <forcer@forcix.cx> wrote:

> Consider a piece of code like this:
>
>     from foo import Foo
>
>     bar = Foo()
>     bar.baz_|_
>
> M-. should go to the definition of the attribute baz of the class Foo.
> But the obvious "identifier at point" is "bar.baz", which by itself
> does not have any relationship to said method without the assignment
> "bar = Foo()", which by itself is also not meaningful without the
> import statement. The only single string that reliably would allow to
> find the correct definition would be "foo.Foo.baz", but I do not think
> that anyone would consider that to be "the identifier at point" here.s

How are you solving (or going to solve) the problem of dynamic/duck typing?

Consider:

```python
def get_content(file_like):
    return file_like.read_|_()
```

Here, in a function that accepts any file-like object, “Find
definition” for read() might mean might mean file.read, or
StringIO.read, or zipfile.ZipExtFile.read, or a read() method of any
of a multitude classes defined in various libraries.

Showing all definitions of read() method of all classes defined in the
transitive closure of modules imported by the main program is
moderately easy but not helpful to the user (e.g. because some classes
are never passed in and their read() implements a different semantic).

Determining which exact subset of read() methods can be available at
the given location in code, I suspect, amounts to solving the Halting
Problem.

Re: Generalizing find-definition

[Jorgen Schaefer]

On Tue, 4 Nov 2014 09:52:28 +0700
Yuri Khan <yuri.v.khan@gmail.com> wrote:

> On Tue, Nov 4, 2014 at 2:55 AM, Jorgen Schaefer <forcer@forcix.cx>
> wrote:
> 
> > Consider a piece of code like this:
> >
> >     from foo import Foo
> >
> >     bar = Foo()
> >     bar.baz_|_
> >
> > [...]
>
> How are you solving (or going to solve) the problem of dynamic/duck
> typing?

Using the Jedi or Rope libraries. They try to guess the identifier at
point (in the example above they can, in your example they can't).

You can try this already today using e.g. Elpy:

https://github.com/jorgenschaefer/elpy

In Elpy, M-. already works as described here.

Regards,
Jorgen

Re: Generalizing find-definition

[Dmitry Gutov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The only benefit would be that M-, is now "go back after M-.", which is
>> what a lot of the packages that redefine M-. do to the key.  Hence the
>> suggestion to swap the two keys, to unify that apparently rather
>> widespread use of M-,.  If Emacs would prefer to keep M-, as is, that is
>> fine with me, too.
>
> I see, so it's compatibility with other packages vs. compatibility with
> etags.el.  Luckily, I think there is no hurry to make this choice.
> So we could poll the users.

I believe ergonomics and frequency of use have also been mentioned. And
if find-tag will provide the "multiple occurences" interface on its own,
there won't be a need for the "next tag" command.

So, +1 from me. You might also take into account that elisp-slime-nav is
a relatively popular package: http://melpa.org/#/elisp-slime-nav

Re: Generalizing find-definition

[Stefan Monnier]

> And if find-tag will provide the "multiple occurences" interface on
> its own, there won't be a need for the "next tag" command.

Good point.  If we don't have a such a global-map "next-tag" command
anymore, then I guess it's OK to change M-, to do the job of M-*.


        Stefan

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Nov 06 2014, Stefan Monnier wrote:

>> And if find-tag will provide the "multiple occurences" interface on
>> its own, there won't be a need for the "next tag" command.
>
> Good point.  If we don't have a such a global-map "next-tag" command
> anymore, then I guess it's OK to change M-, to do the job of M-*.

An idea would be to merge the next-error and next-tag commands. After
all, they are quite similar.

Helmut

Multiple next-error sources

[Jorgen Schaefer]

On Thu, 06 Nov 2014 18:00:24 +0100
Helmut Eller <eller.helmut@gmail.com> wrote:

> On Thu, Nov 06 2014, Stefan Monnier wrote:
> 
> >> And if find-tag will provide the "multiple occurences" interface on
> >> its own, there won't be a need for the "next tag" command.
> >
> > Good point.  If we don't have a such a global-map "next-tag" command
> > anymore, then I guess it's OK to change M-, to do the job of M-*.
> 
> An idea would be to merge the next-error and next-tag commands. After
> all, they are quite similar.

I'm not sure if they are sufficiently similar, but there are other use
cases for something like this. For example, I recently wanted to extend
`next-error' to find flymake highlights.

An idea I had was to replace/extend `next-error-function' with a hook
where the first non-nil return value is used as the location of the
next error.

Regards,
Jorgen

Re: Multiple next-error sources

[Stefan Monnier]

> An idea I had was to replace/extend `next-error-function' with a hook
> where the first non-nil return value is used as the location of the
> next error.

Why?  Can't you already do it with add-function?


        Stefan

Re: Multiple next-error sources

[Jorgen Schaefer]

On Thu, 06 Nov 2014 18:15:40 -0500
Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> > An idea I had was to replace/extend `next-error-function' with a
> > hook where the first non-nil return value is used as the location
> > of the next error.
> 
> Why?  Can't you already do it with add-function?

Sure. There is a single-variable interface already, I thought it would
make sense to extend it.

The advantage of hooks is that it makes it easier for users to
customize the behavior by adding and removing various entries.

Is add-function intended to replace hooks like this in general?

Regards,
Jorgen

Re: Multiple next-error sources

[Stefan Monnier]

> The advantage of hooks is that it makes it easier for users to
> customize the behavior by adding and removing various entries.

While it's true that

    (add-hook 'next-error-functions #'my-function)

is shorter than

    (add-function :before-until next-error-function #'my-function)

I don't think it warrants the addition of a next-error-functions.

If the :before-until is the problematic part, then I guess we should
look for ways to improve that (e.g. a better name, or some way for
a variable to say that :before-until is the default when adding
functions to it?).

> Is add-function intended to replace hooks like this in general?

Somewhat, yes.  I have no intention to go around and replace existing
hooks in the forseeable future (except for those rare hooks that used
with-wrapper-hook), but I'll favor new foo-function over new
foo-functions.


        Stefan

Re: Multiple next-error sources

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

On 11/07/2014 02:59 PM, Stefan Monnier wrote:
>> The advantage of hooks is that it makes it easier for users to
>> customize the behavior by adding and removing various entries.
> 
> While it's true that
> 
>     (add-hook 'next-error-functions #'my-function)
> 
> is shorter than
> 
>     (add-function :before-until next-error-function #'my-function)
> 
> I don't think it warrants the addition of a next-error-functions.
> 
> If the :before-until is the problematic part, then I guess we should
> look for ways to improve that (e.g. a better name, or some way for
> a variable to say that :before-until is the default when adding
> functions to it?).
> 
>> Is add-function intended to replace hooks like this in general?
> 
> Somewhat, yes.  I have no intention to go around and replace existing
> hooks in the forseeable future (except for those rare hooks that used
> with-wrapper-hook), but I'll favor new foo-function over new
> foo-functions.

I strongly dislike this approach. It conflates customization points with
implementation details. Hook variables clearly separate the two ideas.

There's also no buffer-local add-function equivalent.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple next-error sources

[Stefan Monnier]

> I strongly dislike this approach.  It conflates customization points with
> implementation details.  Hook variables clearly separate the two ideas.

foo-function *is* a customization point and not an implementation detail.

That's the difference between using add-function on a foo-function (a
customization point) and using advice-add on some function (some
implementation detail).

> There's also no buffer-local add-function equivalent.

Of course there is:

   (add-function :before-until (local 'next-error-function) #'my-function)


-- Stefan

Re: Multiple next-error sources

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1667 bytes --]

On 11/07/2014 03:55 PM, Stefan Monnier wrote:
>> I strongly dislike this approach.  It conflates customization points with
>> implementation details.  Hook variables clearly separate the two ideas.
> 
> foo-function *is* a customization point and not an implementation detail.
> 
> That's the difference between using add-function on a foo-function (a
> customization point) and using advice-add on some function (some
> implementation detail).

Even the name is unintuitive. Realizing that you can add your own
function to something called "foo-function" (singular) requires a
special kind of comprehension of function composition. "So you're
telling me that I add a function to a function and get a function?"

The many add-function composition modes are useful for advice, but
counterproductive for customization points: the great variety of options
makes it hard to reason about the effect any particular effect. With a
hook, you have a simple list of functions, possibly with a sentinel that
delegates to a global value.

I don't see any compelling reason to avoid conventional hooks. They've
worked for many years. Requiring add-function for some customization and
add-hook for others will only confuse users. It doesn't add any real
power and doesn't make Emacs any better AFAICT.

>> There's also no buffer-local add-function equivalent.
> 
> Of course there is:
> 
>    (add-function :before-until (local 'next-error-function) #'my-function)

Okay, that actually works, including in some corner cases I tested
involving before- and after-functions in both the buffer-local and
global values. My other points still apply.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple next-error sources

[Stefan Monnier]

> The many add-function composition modes are useful for advice, but
> counterproductive for customization points: the great variety of options
> makes it hard to reason about the effect any particular effect.  With a
> hook, you have a simple list of functions, possibly with a sentinel that
> delegates to a global value.

It's a tradeoff, indeed.  It gives you extra flexibility (instead of
the hook specifying that the functions will be composed with
"until-failure", each and every function gets to decide how it's
composed with the others), which of course means more choices to make.

As I said:

   If the :before-until is the problematic part, then I guess we should
   look for ways to improve that (e.g. a better name, or some way for
   a variable to say that :before-until is the default when adding
   functions to it?).

So maybe we should arrange that the "typical" way to compose the
functions for a particular variable be specified along with that
variable, so that you can use (for example)

   (add-function nil next-error-function #'my-function)

and add-function would know to use :before-until.

> I don't see any compelling reason to avoid conventional hooks.

The extra flexibility.

> They've worked for many years.

Obviously not completely, since there are occurrences of foo-function
variables, and for several of those, packages have had to "stash the
previous value, then install their own" on those vars, with latent bugs
when several packages do that on the same var.

> Requiring add-function for some customization and add-hook for others
> will only confuse users.

Yes, I'm not particularly happy about that part, admittedly.


        Stefan

Re: Multiple next-error sources

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1828 bytes --]

On 11/07/2014 06:17 PM, Stefan Monnier wrote:
>> The many add-function composition modes are useful for advice, but
>> counterproductive for customization points: the great variety of options
>> makes it hard to reason about the effect any particular effect.  With a
>> hook, you have a simple list of functions, possibly with a sentinel that
>> delegates to a global value.
> 
> It's a tradeoff, indeed.  It gives you extra flexibility (instead of
> the hook specifying that the functions will be composed with
> "until-failure", each and every function gets to decide how it's
> composed with the others), which of course means more choices to make.
> 
> As I said:
> 
>    If the :before-until is the problematic part, then I guess we should
>    look for ways to improve that (e.g. a better name, or some way for
>    a variable to say that :before-until is the default when adding
>    functions to it?).
> 
> So maybe we should arrange that the "typical" way to compose the
> functions for a particular variable be specified along with that
> variable, so that you can use (for example)

With the existing system, the hook runner gets to specify the "method
combiner" (to use CLOS terminology); with add-function, each hook gets
to specify a different method combiner. It's that possibility that
bothers me, although the lack of a good default increases the risk of
harmful diversity here.

Can you come up with a concrete example of an instance where a
hook-specified method combiner is actually useful?

>    (add-function nil next-error-function #'my-function)
> 
> and add-function would know to use :before-until.
> 
>> I don't see any compelling reason to avoid conventional hooks.
> 
> The extra flexibility.

I'm not yet convinced that the flexibility is worth the cost.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple next-error sources

[Stefan Monnier]

> Can you come up with a concrete example of an instance where a
> hook-specified method combiner is actually useful?

No, currently, there is no variable that is modified via add-function in
different ways in different cases.

I'm sure you can cook up your own artificial example, but currently
practice says this flexibility is indeed not used.

If add-function had existed when post-self-insert-hook had been
introduced, then for sure I would have used a self-insert-function
instead and that would have benefited from such flexibility: some of the
post-self-insert-hook functions currently have to undo what the
self-insert-command did (i.e. they would rather use :around), and others
would really like to know where point was at the beginning of the
command (i.e. they would rather use :around as well).

Of course, a self-insert-function modified with add-function would also
have solved more cleanly the issue of ordering between functions
(currently resolved in an ad-hoc way in
electric--sort-post-self-insertion-hook).  But that would just be
a side-benefit.


        Stefan

Re: Multiple next-error sources

[Jorgen Schaefer]

On Fri, 07 Nov 2014 09:59:12 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > Is add-function intended to replace hooks like this in general?
> 
> Somewhat, yes.  I have no intention to go around and replace existing
> hooks in the forseeable future (except for those rare hooks that used
> with-wrapper-hook), but I'll favor new foo-function over new
> foo-functions.

Thanks for the clarification.

If this is the intended default for Emacs in the future, I would like
to use this in my packages, too. Therefore, I have a few questions.

First, how can I see the sequence of functions being run after
add-function? In the example for next-error, what I was looking at
would have been a list with a few functions from different packages,
such as:

- next-error-if-compile-buffer-is-visible
- next-error-from-flymake
- next-error-from-other-source
- next-error-if-compile-buffer-is-not-visible

With a hook variable, I can C-h v the variable and clearly see what is
happening in which order. I can also easily use setq or delq to change
the order if I find it problematic. Is there a way to see and change the
order of application when using add-function?

Second, also from this example, there would be two functions there by
default:

- next-error-if-compile-buffer-is-visible
- next-error-if-compile-buffer-is-not-visible

New functions likely want to be added in the middle, so visible
compilation buffers take precedence, but non-visible ones are chosen
last. Is there a way using add-function to add a new function in
between two existing functions?

Finally, hook variables can be added to customize to make them more
easily accessible to users. Is there some sort of customize support for
add-function, like there is for hooks/lists of functions?

Regards,
Jorgen

Re: Multiple next-error sources

[Stefan Monnier]

> With a hook variable, I can C-h v the variable and clearly see what is
> happening in which order.

Indeed, this is a downside of add-function.  If you know what you're
looking at, you can kind of figure out what's going on, but it's
definitely in the "I feel like I'm decrypting rather than
reading" category.

Of course, you can use advice-function-mapc to extract the info and turn
it into a normal list, but I didn't bother to write such a function, nor
to hook it into C-h v.  Patches welcome ;-)

> I can also easily use setq or delq to change
> the order if I find it problematic.

You can "delq" with remove-function.  I don't think there's a real
equivalent of "setq", OTOH.

> New functions likely want to be added in the middle, so visible
> compilation buffers take precedence, but non-visible ones are chosen
> last.  Is there a way using add-function to add a new function in
> between two existing functions?

You can specify a `depth' property, yes.

> Finally, hook variables can be added to customize to make them more
> easily accessible to users. Is there some sort of customize support for
> add-function, like there is for hooks/lists of functions?

Currently, no.


        Stefan

Re: Multiple next-error sources

[Alan Mackenzie]

Hello, Stefan.

On Fri, Nov 07, 2014 at 09:59:12AM -0500, Stefan Monnier wrote:
> > The advantage of hooks is that it makes it easier for users to
> > customize the behavior by adding and removing various entries.

> While it's true that

>     (add-hook 'next-error-functions #'my-function)

> is shorter than

>     (add-function :before-until next-error-function #'my-function)

> I don't think it warrants the addition of a next-error-functions.

Why not?

> If the :before-until is the problematic part, then I guess we should
> look for ways to improve that (e.g. a better name, or some way for
> a variable to say that :before-until is the default when adding
> functions to it?).

> > Is add-function intended to replace hooks like this in general?

> Somewhat, yes.  I have no intention to go around and replace existing
> hooks in the forseeable future (except for those rare hooks that used
> with-wrapper-hook), but I'll favor new foo-function over new
> foo-functions.

Why?  I'm adding my voice to the clamour of dissent.

It would seem that the use of single functions, with `add-function' is
inferior to the conventional hook mechanism in every way.  What am I
missing?  In addition to the things cited by Daniel, there's:

(i) the danger (near certainty) that somebody is going to use `setq'
rather than `add-function' to configure it;

(ii) the additional incompatibility with other Emacsen;

(iii) the difficulty (or perhaps clumsiness) in looking at the contents
of an advised function.  There would seem to be nothing equivalent to
M-: after-change-functions.

So why are you changing from the conventional hook mechanism, which works
so well?  What is the advantage of the new scheme.

Incidentally, I had a look at the documentation for add-advice, and
there's a problem with it.  "Advice" in English has no plural - there's
no such word as "advices".  If it's necessary to emphasize the plurality,
then "pieces of advice" can be used.  There's no such thing as "an
advice", rather you'd say "some advice".  It's a bit like you wouldn't
refer to a lake as "a big water"; you'd say it contains "a lot of water".
I think there's a term in linguistics for such a word, but I don't know
it off hand.

Incidentally 2, the verb corresponding to the noun "advice" is "to
advise".

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Multiple next-error sources

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 2839 bytes --]

On 11/07/2014 04:55 PM, Alan Mackenzie wrote:
> Hello, Stefan.
> 
> On Fri, Nov 07, 2014 at 09:59:12AM -0500, Stefan Monnier wrote:
>>> The advantage of hooks is that it makes it easier for users to
>>> customize the behavior by adding and removing various entries.
> 
>> While it's true that
> 
>>     (add-hook 'next-error-functions #'my-function)
> 
>> is shorter than
> 
>>     (add-function :before-until next-error-function #'my-function)
> 
>> I don't think it warrants the addition of a next-error-functions.
> 
> Why not?
> 
>> If the :before-until is the problematic part, then I guess we should
>> look for ways to improve that (e.g. a better name, or some way for
>> a variable to say that :before-until is the default when adding
>> functions to it?).
> 
>>> Is add-function intended to replace hooks like this in general?
> 
>> Somewhat, yes.  I have no intention to go around and replace existing
>> hooks in the forseeable future (except for those rare hooks that used
>> with-wrapper-hook), but I'll favor new foo-function over new
>> foo-functions.
> 
> Why?  I'm adding my voice to the clamour of dissent.
> 
> It would seem that the use of single functions, with `add-function' is
> inferior to the conventional hook mechanism in every way.  What am I
> missing?  In addition to the things cited by Daniel, there's:
> 
> (i) the danger (near certainty) that somebody is going to use `setq'
> rather than `add-function' to configure it;

The same critique applies to regular hooks, doesn't it?

> (ii) the additional incompatibility with other Emacsen;

I'm not sure that compatibility with other Emacsen is as important as it
once was. AIUI, GNU Emacs is receiving the vast majority of development
effort.

> (iii) the difficulty (or perhaps clumsiness) in looking at the contents
> of an advised function.  There would seem to be nothing equivalent to
> M-: after-change-functions.
> 
> So why are you changing from the conventional hook mechanism, which works
> so well?  What is the advantage of the new scheme.
> 
> Incidentally, I had a look at the documentation for add-advice, and
> there's a problem with it.  "Advice" in English has no plural - there's
> no such word as "advices".  If it's necessary to emphasize the plurality,
> then "pieces of advice" can be used.  There's no such thing as "an
> advice", rather you'd say "some advice".  It's a bit like you wouldn't
> refer to a lake as "a big water"; you'd say it contains "a lot of water".
> I think there's a term in linguistics for such a word, but I don't know
> it off hand.

I think "advise" works like "code" in the software sense and
"furniture". The term is "mass noun".

> Incidentally 2, the verb corresponding to the noun "advice" is "to
> advise".

Isn't English fun?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple next-error sources

[Alan Mackenzie]

Hello, Daniel.

On Fri, Nov 07, 2014 at 05:10:29PM +0000, Daniel Colascione wrote:
> On 11/07/2014 04:55 PM, Alan Mackenzie wrote:

> > It would seem that the use of single functions, with `add-function' is
> > inferior to the conventional hook mechanism in every way.  What am I
> > missing?  In addition to the things cited by Daniel, there's:

> > (i) the danger (near certainty) that somebody is going to use `setq'
> > rather than `add-function' to configure it;

> The same critique applies to regular hooks, doesn't it?

:-)  So it does!  What I was confused about is foo-function.  I think
this is going to be a defun in the future, whereas up to now it's always
been a defvar.  This is confusing.  So whereas you'd use "(setq
c-backspace-function 'foobar)", and use `funcall' to invoke it, you'd say
"(add-function 'foo-function 'foobar)" (or whatever), and just call
`foo-function' as a function.

> > (ii) the additional incompatibility with other Emacsen;

> I'm not sure that compatibility with other Emacsen is as important as it
> once was. AIUI, GNU Emacs is receiving the vast majority of development
> effort.

It may be less important than it was, but that's no reason for dismissing
it altogether.

> > (iii) the difficulty (or perhaps clumsiness) in looking at the contents
> > of an advised function.  There would seem to be nothing equivalent to
> > M-: after-change-functions.

> > So why are you changing from the conventional hook mechanism, which works
> > so well?  What is the advantage of the new scheme.

> > Incidentally, I had a look at the documentation for add-advice, and
> > there's a problem with it.  "Advice" in English has no plural - there's
> > no such word as "advices".  If it's necessary to emphasize the plurality,
> > then "pieces of advice" can be used.  There's no such thing as "an
> > advice", rather you'd say "some advice".  It's a bit like you wouldn't
> > refer to a lake as "a big water"; you'd say it contains "a lot of water".
> > I think there's a term in linguistics for such a word, but I don't know
> > it off hand.

> I think "advise" ...

"advice" ??

> ... works like "code" in the software sense and "furniture". The term
> is "mass noun".

Thanks!  "Furniture" is indeed a better example than "water".

> > Incidentally 2, the verb corresponding to the noun "advice" is "to
> > advise".

> Isn't English fun?

Indeed it is!  You get to appreciate it especially when you live in a
place where they don't speak your native language (much).

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Multiple next-error sources

[Dmitry Gutov]

Alan Mackenzie <acm@muc.de> writes:

>> > (i) the danger (near certainty) that somebody is going to use `setq'
>> > rather than `add-function' to configure it;
>
>> The same critique applies to regular hooks, doesn't it?
>
> :-)  So it does!  What I was confused about is foo-function.  I think
> this is going to be a defun in the future, whereas up to now it's always
> been a defvar.

Will it? In that case, it probably should be named differently, like
foo-default (next-error-default, in the current example).

As long as there is a foo-function custom var, the critique applies.

Re: Multiple next-error sources

[Stefan Monnier]

>> While it's true that
>> (add-hook 'next-error-functions #'my-function)
>> is shorter than
>> (add-function :before-until next-error-function #'my-function)
>> I don't think it warrants the addition of a next-error-functions.
> Why not?

I'd return the question: given the above, why add next-error-functions?


        Stefan

Re: Multiple next-error sources

[Alan Mackenzie]

Hello, Stefan.

On Fri, Nov 07, 2014 at 01:08:57PM -0500, Stefan Monnier wrote:
> >> While it's true that
> >> (add-hook 'next-error-functions #'my-function)
> >> is shorter than
> >> (add-function :before-until next-error-function #'my-function)
> >> I don't think it warrants the addition of a next-error-functions.
> > Why not?

> I'd return the question: given the above, why add next-error-functions?

Because "(add-hook 'next-error-functions #'my-function)" is shorter than
the `add-function' variant, along with all the other good reasons which
have been posted in this thread.

Let me repeat my question: in what respect is the `add-function' way of
doing things superior to the conventional hook mechanism?  If the answer
is "in no respect", then I would suggest that the project continue to
use hooks in preference to `add-function' functions.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Multiple next-error sources

[Stefan Monnier]

> Because "(add-hook 'next-error-functions #'my-function)" is shorter than
> the `add-function' variant,

The text I wrote and you quoted already said that it was shorter but not
enough to warrant an additional variable.  So you think saving 16
characters is worth an additional hook?
Based on that, I guess I should just change the naming conventions:

   (add-hook 'next-error-functions #'my-function)
   (add-f :before-until next-error-f #'my-f)

Look ma!  We should change all hooks to be plain foo-f variables!

> along with all the other good reasons which have been posted in
> this thread.

None of those reasons seem to apply the question of whether it's worth
adding a next-error-functions hook (remember: next-error-function exists
already, since long before add-function was born).

> Let me repeat my question: in what respect is the `add-function' way of
> doing things superior to the conventional hook mechanism?  If the answer
> is "in no respect", then I would suggest that the project continue to
> use hooks in preference to `add-function' functions.

At this point I feel like you're confusing me with an idiot.


        Stefan

Re: Multiple next-error sources

[Alan Mackenzie]

Hello, Stefan.

On Fri, Nov 07, 2014 at 01:48:25PM -0500, Stefan Monnier wrote:
> > Because "(add-hook 'next-error-functions #'my-function)" is shorter than
> > the `add-function' variant,

> The text I wrote and you quoted already said that it was shorter but not
> enough to warrant an additional variable.  So you think saving 16
> characters is worth an additional hook?

It saves one parameter, that `:before-until', which is difficult to
understand.

> None of those reasons seem to apply the question of whether it's worth
> adding a next-error-functions hook (remember: next-error-function exists
> already, since long before add-function was born).

> > Let me repeat my question: in what respect is the `add-function' way of
> > doing things superior to the conventional hook mechanism?  If the answer
> > is "in no respect", then I would suggest that the project continue to
> > use hooks in preference to `add-function' functions.

> At this point I feel like you're confusing me with an idiot.

Sorry, I don't mean to give that impression.

Anyhow, you've answered my question in a post to Daniel.  It's all about
increased flexibility.  So, thanks!

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> M-. RET  does "find the callers of a symbol at point", AFAICT.
>
> Duh, sorry for being dense, of course it doesn't.  etags does not offer
> the needed info, so the etags.el UI doesn't have such a thing.

In emacs -Q, M-. is 'find-tag', which prompts for a tag with the default
being the identifer at point, and goes to the first occurrence of that
identifier in the tags table. Then tags-loop-continue goes to the next
occurrence.

The occurrences will include the calling locations; perhaps that is what
was meant here.

In other situations, "find the callers" would mean go to the start or
declaration of the calling functions, not the point of the call.

ada-mode provides "show all references", which is the same as "show all
calls" for functions. It does not currently have "show all callers".

> So on the UI side we mostly have:
> - "jump to *the* definition of thing at point".
> - "list definitions of thing at point".

    These two should be combined; only show the list if it has more than
    one item. The point is that in most cases there is only one, so
    showing the list is just an annoyance.
    
> - "list uses of thing at point".

    This should show the list if there is only one; that's not typical,
    and is important information.
    
> - "return to buffer/position before the last jump".

I'm not clear this should be separate from the mark ring (push-mark,
pop-mark).

> And on the backend side we have:
> - identifier-at-point-function

    This probably does not need to be separate; the other functions will
    call the equivalent backend function.

> - find-definition-function (with an argument to decide whether we want
>   a whole list or just the best/first candidate).

    I don't think we need that argument. Instead, it could take an
    optional argument 'identifier', which would be used instead of
    indentifier-at-point.
    
> - find-uses-function

> Etags.el currently offers some additional functionality:
> - jump to definition of any identifier, with TAB-completion.

    This implies that the backend has to provide the list of identifiers
    for completion; that's a new backend function 'complete-identifier'.

    ada-mode doesn't have that now, but it could be useful.

    It could also get very slow; the list of identifiers can be very
    large. I guess that's no different from the current tags usage.
    
> - jump to file (among those listed in the TAGS file), tho currently
>   without TAB-completion.

    Another backend function; 'complete-file-name'.

    Or it could be 'project-directories' and 'project-extensions', to
    be used with ff-get-file.

    ada-mode uses compilation-search-path for 'project-directories', and
    ada-body-suffixes, ada-spec-suffixes for 'project-extensions.

    ada-mode does not currently provide jump to file with prompt, only
    jump to file derived from identifier at point. Prompt would be
    useful.
    
> And the second one is also provided by things like projectile (among
> many other thingies), tho it seems that none of the bundled Emacs
> packages provide a good solution for that common need.

I think ff-get-file is good. But maybe others want a more restricted list?

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

>> And the second one is also provided by things like projectile (among
>> many other thingies), tho it seems that none of the bundled Emacs
>> packages provide a good solution for that common need.
> I think ff-get-file is good. But maybe others want a more restricted list?

Not only ff-get-file is a function, not a command, but I don't think it
provides the same functionality.  E.g. it doesn't look recursively
inside subdirs.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> And the second one is also provided by things like projectile (among
>>> many other thingies), tho it seems that none of the bundled Emacs
>>> packages provide a good solution for that common need.
>> I think ff-get-file is good. But maybe others want a more restricted list?
>
> Not only ff-get-file is a function, not a command, 

It could be wrapped; I was referring to the functionality.

> but I don't think it
> provides the same functionality.  E.g. it doesn't look recursively
> inside subdirs.

the function we are talking about is:

> - jump to file (among those listed in the TAGS file)

So looking recursively inside subdirs is wrong.

The TAGS file contains a list of file:line:column (among other things).
In this context, we only care about the file names.

That list of file names is the source code for the current project (for
some loose definition of 'project').

So the function we are defining is 'goto-file-in-project', with the user
prompted for the file name.

In ada-mode, the definition of 'file in project' is the combination of
compilation-search-path with ada-spec-suffixes, ada-body-suffixes. I
believe EDE projects are similar.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

>> - jump to file (among those listed in the TAGS file)
> So looking recursively inside subdirs is wrong.

The TAGS file can (and in several cases does) contain data about files
in subdirectories.  Basically "jump to file listed in TAGS" lets (or
wants to do) you jump to any file in the current project.

> So the function we are defining is 'goto-file-in-project', with the user
> prompted for the file name.

Exactly.  And I don't think ff-get-file quite does it.

> In ada-mode, the definition of 'file in project' is the combination of
> compilation-search-path with ada-spec-suffixes, ada-body-suffixes. I
> believe EDE projects are similar.

That sounds right.


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

[-- Attachment #1: Type: text/plain, Size: 1498 bytes --]

On Sun, 02 Nov 2014 10:34:28 -0500
Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > M-. is bound to a new command `find-definition', which primarily
> > calls the value of a new variable `find-definition-function' (by
> > default a wrapper around `find-tag' to keep the current
> > functionality intact). `find-definition' also keeps track of the
> > tag ring, so this would move `find-tag-marker-ring' and related
> > functionality out of etags.el, too.
> 
> Yes, this sounds great.
>
> [...]
>
> > Comments?
> 
> I'm all for it,

Some initial patch attached. Is this what you had in mind when you said
the above? Anything I should change to make it fit into Emacs better? I
do not know much about Emacs coding conventions.

Currently missing:

- find-definition does not store locations in the tag ring yet
  (ran out of time and wanted to post the initial patch for feedback)
- The whole "find uses" interface
- etags.el integration
- emacs-lisp-mode integration

This also does define a minor mode instead of changing the global key
bindings. I think in the final version it should replace the key
binding definitions done in etags.el. Is this correct? Do I need to
hook it up elsewhere in the build process?

Public git repo for those who find that easier to read / clone (nothing
there that isn't in the attached patch):

https://github.com/jorgenschaefer/emacs/tree/find-definition
https://github.com/jorgenschaefer/emacs/commit/ecac500ac7dee3095863df23c4cd661ba62e2187

Regards,
Jorgen

[-- Attachment #2: 0001-lisp-progmodes-find-definition.el-New-file.patch --]
[-- Type: text/x-patch, Size: 16839 bytes --]

From ecac500ac7dee3095863df23c4cd661ba62e2187 Mon Sep 17 00:00:00 2001
From: Jorgen Schaefer <contact@jorgenschaefer.de>
Date: Mon, 17 Nov 2014 20:43:57 +0100
Subject: [PATCH] * lisp/progmodes/find-definition.el: New file. *
 test/automated/find-definition-test.el: New file.

---
 lisp/ChangeLog                         |    4 +
 lisp/progmodes/find-definition.el      |  194 +++++++++++++++++++++++++++++
 test/ChangeLog                         |    4 +
 test/automated/find-definition-test.el |  208 ++++++++++++++++++++++++++++++++
 4 files changed, 410 insertions(+)
 create mode 100644 lisp/progmodes/find-definition.el
 create mode 100644 test/automated/find-definition-test.el

diff --git a/lisp/ChangeLog b/lisp/ChangeLog
index 912b69a..0334bb4 100644
--- a/lisp/ChangeLog
+++ b/lisp/ChangeLog
@@ -1,3 +1,7 @@
+2014-11-17  Jorgen Schaefer <contact@jorgenschaefer.de>
+
+	* progmodes/find-definition.el: New file.
+
 2014-11-17  Lars Magne Ingebrigtsen  <larsi@gnus.org>
 
 	* bindings.el (search-map): Move `eww-search-words' to `M-s M-w'.
diff --git a/lisp/progmodes/find-definition.el b/lisp/progmodes/find-definition.el
new file mode 100644
index 0000000..d9846e7
--- /dev/null
+++ b/lisp/progmodes/find-definition.el
@@ -0,0 +1,194 @@
+;;; find-definition.el --- Find definition at point  -*- lexical-binding: t -*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; Author: Jorgen Schaefer <contact@jorgenschaefer.de>
+;; Keywords: tools
+
+;; This file is part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or
+;; modify it under the terms of the GNU General Public License
+;; as published by the Free Software Foundation; either version 3
+;; of the License, or (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;; Code:
+
+(require 'ring)
+
+(defgroup find-definition nil "Finding definitions of things at point."
+  :group 'tools)
+
+(defcustom find-definition-marker-ring-length 16
+  "Length of marker rings `find-definition-marker-ring'."
+  :group 'find-definition
+  :type 'integer)
+
+(defvar find-definition-function nil
+  "The function `find-definition' calls to find the definition.
+
+Will be called with no arguments with point at the location of
+the thing to find the definition for. It should return a list
+with each element being a list of one to three elements. The
+first element should be the file name, the second the
+line (defaulting to 1) and the third the column (defaulting to
+0).")
+
+(defvar find-definition-identifier-function nil
+  "Find the definition of a named identifier.
+
+Will be called with the result of prompting the user for a
+completion using `find-definition-completion-table', and should
+return a list like `find-definition-function'.")
+
+(defvar find-definition-identifier-completion-table nil
+  "The completion table to complete known symbols.
+
+Will be passed as COLLECTION to `completing-read'.")
+
+(defvar find-definition-marker-ring
+  (make-ring find-definition-marker-ring-length)
+  "Ring of positions visited by `find-definition'.")
+
+(defvar find-definition-mode-map
+  (let ((map (make-sparse-keymap)))
+    (define-key map (kbd "M-.") 'find-definition)
+    (define-key map (kbd "C-x 4 .") 'find-definition-other-window)
+    (define-key map (kbd "C-x 5 .") 'find-definition-other-frame)
+    ;; (define-key map (kbd "M-_") 'find-definition-uses)
+    (define-key map (kbd "M-,") 'find-definition-goto-last-position)
+    map)
+  "The key map for `find-definition-mode'.")
+
+;;;###autoload
+(define-minor-mode find-definition-mode
+  "Minor mode to provide some key bindings to find definitions.
+
+\\{find-definition-mode-map}"
+  :keymap 'find-definition-mode)
+
+;;;###autoload
+(defun find-definition (&optional ask)
+  "Go to the definition of the thing at point.
+
+If the definition can not be found, or with a prefix argument,
+prompt for a symbol to use."
+  (interactive "P")
+  (switch-to-buffer (find-definition--noselect ask)))
+
+;;;###autoload
+(defun find-definition-other-window (&optional ask)
+  "Display the definition of the thing at point in another window.
+
+If the definition can not be found, or with a prefix argument,
+prompt for a symbol to use."
+  (interactive "P")
+  (switch-to-buffer-other-window (find-definition--noselect ask)))
+
+;;;###autoload
+(defun find-definition-other-frame (&optional ask)
+  "Display the definition of the thing at point in another frame.
+
+If the definition can not be found, or with a prefix argument,
+prompt for a symbol to use."
+  (interactive "P")
+  (switch-to-buffer-other-frame (find-definition--noselect ask)))
+
+(defun find-definition--noselect (&optional ask)
+  "Internal function for `find-definition'.
+
+Does all the work, but returns the buffer instead of displaying
+it."
+  (let* ((locations (when (not ask)
+                      (funcall find-definition-function))))
+    (cond
+     (locations
+      (find-definition--find-locations locations))
+     ((and find-definition-identifier-completion-table
+           find-definition-identifier-function)
+      (let* ((identifier (completing-read
+                          "Find definition: "
+                          find-definition-identifier-completion-table
+                          nil t))
+             (locations (funcall find-definition-identifier-function
+                                 identifier)))
+        (find-definition--find-locations locations)))
+     (t
+      (error "Can't find the definition of the thing at point")))))
+
+(defun find-definition--find-locations (locations)
+  "Go to the location in LOCATIONS.
+
+If there is exactly one location, go directly there. Otherwise,
+prompt the user for a location choice."
+  (if (null (cdr locations))
+      ;; Exactly one definition
+      (let* ((location (car locations))
+             (filename (elt location 0))
+             (line (or (elt location 1)
+                       1))
+             (col (or (elt location 2)
+                      0))
+             (buf (find-file-noselect filename)))
+        (with-current-buffer buf
+          (widen)
+          (goto-char (point-min))
+          (forward-line (- line 1))
+          (forward-char col))
+        buf)
+    ;; More than one definition
+    (let ((outbuf (get-buffer-create "*Definitions*"))
+          (dir default-directory)
+          (inhibit-read-only t))
+      (with-current-buffer outbuf
+        (erase-buffer)
+        (setq default-directory dir)
+        (compilation-mode)
+        (dolist (location locations)
+          (let* ((filename (elt location 0))
+                 (line (or (elt location 1)
+                           1))
+                 (col (or (elt location 2)
+                          0))
+                 (buffer (find-buffer-visiting filename))
+                 (line-string
+                  (when buffer
+                    (with-current-buffer buffer
+                      (save-excursion
+                        (save-restriction
+                          (widen)
+                          (goto-char (point-min))
+                          (forward-line (- line 1))
+                          (buffer-substring (line-beginning-position)
+                                            (line-end-position))))))))
+            (insert (format "%s:%s:%s:%s\n"
+                            filename line col
+                            (or line-string
+                                "")))))
+        (goto-char (point-min)))
+      outbuf)))
+
+;;;###autoload
+(defun find-definition-goto-last-position ()
+  "Pop back to where \\[find-definition] was last invoked."
+  (interactive)
+  (when (ring-empty-p find-definition-marker-ring)
+    (error "No previous locations for find-definition invocation"))
+  (let ((marker (ring-remove find-definition-marker-ring)))
+    (switch-to-buffer (or (marker-buffer marker)
+                          (error "The marked buffer has been deleted")))
+    (goto-char (marker-position marker))
+    (set-marker marker nil nil)))
+
+(provide 'find-definition)
+;;; find-definition.el ends here
diff --git a/test/ChangeLog b/test/ChangeLog
index fb00410..217672e 100644
--- a/test/ChangeLog
+++ b/test/ChangeLog
@@ -1,3 +1,7 @@
+2014-11-17  Jorgen Schaefer <contact@jorgenschaefer.de>
+
+	* automated/find-definition-test.el: New file.
+
 2014-11-17  Glenn Morris  <rgm@gnu.org>
 
 	* automated/occur-tests.el (occur-test-case, occur-test-create):
diff --git a/test/automated/find-definition-test.el b/test/automated/find-definition-test.el
new file mode 100644
index 0000000..920eb30
--- /dev/null
+++ b/test/automated/find-definition-test.el
@@ -0,0 +1,208 @@
+;;; find-definition-test.el --- Test suite for find-definition.el -*- lexical-binding: t -*-
+
+;; Copyright (C) 2014  Jorgen Schaefer <contact@jorgenschaefer.de>
+
+;; This program is free software; you can redistribute it and/or
+;; modify it under the terms of the GNU General Public License
+;; as published by the Free Software Foundation; either version 3
+;; of the License, or (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;; Code:
+
+(require 'ert)
+(require 'find-definition)
+
+
+(defmacro find-definition-with-temp-file (variable &rest body)
+  "Create a temporary file, bind it to VARIABLE, and evaluated BODY.
+
+Delete the file after BODY finishes."
+  (declare (indent 1))
+  `(let ((,variable (make-temp-file "find-definition-test-")))
+     (unwind-protect
+         (progn ,@body)
+       (delete-file ,variable))))
+
+;;;;;;;;;;;;;;;;;;;
+;;; find-definition
+
+(ert-deftest find-definition ()
+  ;; It should go to the definition if there is exactly one.
+  (find-definition-with-temp-file filename
+    (with-temp-file filename
+      (insert "Hello\n"
+              "World\n"))
+    (let ((find-definition-function (lambda ()
+                                      (list (list filename 2 3)))))
+
+      (find-definition)
+
+      (should (equal (buffer-file-name) filename))
+      (should (looking-at "ld")))
+    (kill-buffer))
+
+  ;; If the backend function can't find a definition for the thing at
+  ;; point, find-definition should prompt the user for a symbol
+  ;; completed by a backend-provided completion table. This symbol
+  ;; then is passed to a backend function to get the location.
+  (find-definition-with-temp-file filename
+    (cl-letf* ( ;; User function definitions
+               (find-definition-function (lambda ()
+                                           nil))
+               (find-definition-identifier-completion-table 'test-table)
+               (fdsf-identifier nil)
+               (find-definition-identifier-function
+                (lambda (sym)
+                  (setq fdsf-identifier sym)
+                  (list (list filename 1 0))))
+               ;; Mocking
+               (cr-collection nil)
+               ((symbol-function 'completing-read)
+                (lambda (prompt collection &rest args)
+                  (setq cr-collection collection)
+                  "test-symbol")))
+
+      (find-definition)
+
+      (should (equal cr-collection 'test-table))
+      (should (equal fdsf-identifier "test-symbol"))
+      (should (equal (buffer-file-name) filename))))
+
+  ;; Do the same with a prefix argument.
+  (find-definition-with-temp-file filename
+    (cl-letf* ( ;; User function definitions
+               (find-definition-function (lambda ()
+                                           (error "Should not be called")))
+               (find-definition-identifier-completion-table 'test-table)
+               (find-definition-identifier-function
+                (lambda (sym)
+                  (list (list filename 1 0))))
+               ;; Mocking
+               (cr-collection nil)
+               ((symbol-function 'completing-read)
+                (lambda (prompt collection &rest args)
+                  (setq cr-collection collection)
+                  "test-symbol")))
+
+      (find-definition '(4))
+
+      (should (equal cr-collection 'test-table))))
+
+  ;; Without a completion table and no identifier at point, the
+  ;; function should throw an error.
+  (cl-letf* ((find-definition-function (lambda ()
+                                         nil))
+             (find-definition-identifier-completion-table nil)
+             (find-definition-identifier-function nil))
+
+    (should-error
+     (find-definition)))
+
+  ;; Without a completion table and a prefix argument, the function
+  ;; should throw an error as well
+  (cl-letf* ((find-definition-function (lambda ()
+                                         (error "Should not be called")))
+             (find-definition-identifier-completion-table nil)
+             (find-definition-identifier-function nil))
+
+    (should-error
+     (find-definition '(4))))
+
+  ;; It should pop up a selection dialog if there is more than one
+  ;; definition.
+  (find-definition-with-temp-file filename
+    (with-temp-file filename
+      (insert "Hello\n"
+              "World\n"))
+    (let ((find-definition-function (lambda ()
+                                      (list (list filename 1)
+                                            (list filename 2)))))
+
+      (find-definition)
+
+      (should (equal (buffer-name) "*Definitions*"))
+      (should (eq major-mode 'compilation-mode))))
+  )
+
+;;;;;;;;;;;;;;;;;;;;;;;;
+;;; find-definition-uses
+
+;; - find-definition-uses should go the use if there is exactly one.
+;; - find-definition-uses should pop up a selection dialog if there is
+;;   more than one use.
+;; - find-definition-uses should prompt for name if there is no definition.
+;; - find-definition-uses should prompt for name with prefix argument.
+
+;; - Provide a new function, etags-find-definition
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;;; find-definition-goto-last-position
+
+(ert-deftest find-definition-goto-last-position ()
+  ;; It should move to the last position in the same buffer
+  (with-temp-buffer
+    (let* ((find-definition-marker-ring (make-ring 5))
+           (start (point))
+           (marker (make-marker)))
+      (set-marker marker start)
+      (ring-insert find-definition-marker-ring marker)
+      (insert "Bla bla\n")
+      (should (not (= (point) start)))
+      (find-definition-goto-last-position)
+      (should (= (point) start))))
+
+  ;; It should move to the last position in another buffer
+  (with-temp-buffer
+    (let* ((find-definition-marker-ring (make-ring 5))
+           (start (point))
+           (start-buffer (current-buffer))
+           (marker (make-marker)))
+      (set-marker marker start start-buffer)
+      (ring-insert find-definition-marker-ring marker)
+      (with-temp-buffer
+        (find-definition-goto-last-position)
+        (should (equal (current-buffer) start-buffer))
+        (should (= (point) start)))))
+
+  ;; It should should be interactive
+  (should (interactive-form 'find-definition-goto-last-position))
+
+  ;; It should raise an error when the marker ring is empty
+  (let ((find-definition-marker-ring (make-ring 1)))
+    (should-error (find-definition-goto-last-position)))
+
+  ;; It should raise an error if the original buffer is deleted
+  (let* ((find-definition-marker-ring (make-ring 5))
+         (marker (make-marker)))
+    (with-temp-buffer
+      (set-marker marker (point))
+      (ring-insert find-definition-marker-ring marker))
+    (should-error
+     (find-definition-goto-last-position)))
+
+  ;; It should reset the marker
+  ;;
+  ;; Why? This is copied from pop-to-mark, but why would this be
+  ;; needed?
+  (let* ((find-definition-marker-ring (make-ring 5))
+         (marker (make-marker)))
+    (with-temp-buffer
+      (set-marker marker (point))
+      (ring-insert find-definition-marker-ring marker)
+      (find-definition-goto-last-position)
+      (should (null (marker-position marker)))
+      (should (null (marker-buffer marker)))))
+  )
+
+(provide 'find-definition-test)
+;;; find-definition-test.el ends here
-- 
1.7.10.4

Re: Generalizing find-definition

[Stephen Leake]

Jorgen Schaefer <forcer@forcix.cx> writes:

> This also does define a minor mode instead of changing the global key
> bindings. I think in the final version it should replace the key binding
> definitions done in etags.el. Is this correct?

+1

> Do I need to hook it up elsewhere in the build process?

Not sure what you are asking here; the keybindings need to be autoloaded
as in etags.el.


> +(defvar find-definition-function nil
> +(defvar find-definition-identifier-function nil

These variables will be set by the major mode, so they should be
buffer-local; use defvar-local.


> +(defvar find-definition-marker-ring
> +  (make-ring find-definition-marker-ring-length)
> +  "Ring of positions visited by `find-definition'.")

This variable might want to be mode-specific, but not buffer-local; I'm
not sure how to accomplish that.

-- 
-- Stephe

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Nov 18 2014, Stephen Leake wrote:

>> +(defvar find-definition-marker-ring
>> +  (make-ring find-definition-marker-ring-length)
>> +  "Ring of positions visited by `find-definition'.")
>
> This variable might want to be mode-specific, but not buffer-local; I'm
> not sure how to accomplish that.

I think this should be global and not mode-specific.  Making it global
makes it easier to switch between languages, e.g. from ELisp to C and
back.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

Helmut Eller <eller.helmut@gmail.com> writes:

> I think this should be global and not mode-specific.  Making it global
> makes it easier to switch between languages, e.g. from ELisp to C and
> back.

Global sounds better to me, too. But making storage flexible would be
even better. Some might prefer it to be project-local.

Personally, I'd rather make it local to a window, like implemented here:
https://github.com/dgutov/point-stack/blob/master/point-stack.el#L83

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Nov 18 2014, Stephen Leake wrote:

>> +(defvar find-definition-function nil
>> +(defvar find-definition-identifier-function nil

Is it possible to use EIEIO instead?  I have a hunch that a "backend"
will need many "methods" in the long run.  My idea is that a mode
creates a subclass of something like a find-definition-backend class and
defines/overrides methods as appropriate for the mode.  The backend
object could then be stored in a buffer local variable.  Using EIEIO
classes and methods would arguably be clearer and more flexible than
defining a bunch of FOO-function variables.

In pseudocode:

;; interface of backend
(defclass find-definition-backend () ())
(defgeneric find-definition-identifier (backend))
(defgeneric find-definitions (backend identifier))

;; implementation of Elisp backend
(defclass elisp-backend (find-definition-backend) ())

(defmethod find-definition-identifier ((backend elisp-backend))
  (symbol-at-point))

(defmethod find-definitions ((backend elisp-backend) symbol)
  ...)

;; initialize buffer local backend variable in for elisp-mode:
(set (make-local-variable 'find-definition-backend)
     (make-instance 'elisp-backend))

Helmut

Re: Generalizing find-definition

[Stefan Monnier]

>> +(defvar find-definition-function nil
>> +(defvar find-definition-identifier-function nil
> These variables will be set by the major mode, so they should be
> buffer-local; use defvar-local.

Bad idea: if the major modes set those with `setq' rather than with
`setq-local', they'll end up setting the global value if the `setq'
happens to be run before loading find-definition.el (which is quite
possible since find-definition.el will typically be loaded via an
autoload rather than via a `require').


        Stefan

Re: Generalizing find-definition

[Ivan Shmakov]

>>>>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

 >>> +(defvar find-definition-function nil
 >>> +(defvar find-definition-identifier-function nil

 >> These variables will be set by the major mode, so they should be
 >> buffer-local; use defvar-local.

 > Bad idea: if the major modes set those with `setq' rather than with
 > `setq-local', they'll end up setting the global value if the `setq'
 > happens to be run before loading find-definition.el (which is quite
 > possible since find-definition.el will typically be loaded via an
 > autoload rather than via a `require').

	Can’t we ;;;###autoload the defvar-local forms themselves then?

-- 
FSF associate member #7257  np. Benediction — Jami Sieber   … B6A0 230E 334A

Re: Generalizing find-definition

[Stefan Monnier]

> 	Can’t we ;;;###autoload the defvar-local forms themselves then?

Why would we want to?  Just tell the major-modes to use `setq-local',
like they do for all other variables.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>> +(defvar find-definition-function nil
>>> +(defvar find-definition-identifier-function nil
>> These variables will be set by the major mode, so they should be
>> buffer-local; use defvar-local.
>
> Bad idea: if the major modes set those with `setq' rather than with
> `setq-local', they'll end up setting the global value if the `setq'
> happens to be run before loading find-definition.el (which is quite
> possible since find-definition.el will typically be loaded via an
> autoload rather than via a `require').

This _should_ be caught by a byte-compiler warning
("find-definition-function not known"). So I'm ok with requiring major
modes to either have "(require find-definition)" or use setq-local.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> This _should_ be caught by a byte-compiler warning
> ("find-definition-function not known").

I wouldn't bet on it.

> So I'm ok with requiring major modes to either have "(require
> find-definition)" or use setq-local.

The normal course of event for major mode is to set variables via
`setq-local' (or its equivalent for use on older Emacsen).  So there's
no good reason to use make-variable-buffer-local for that; it'd just be
asking for trouble.


        Stefan

Re: Generalizing find-definition

[Stefan Monnier]

> - find-definition does not store locations in the tag ring yet
>   (ran out of time and wanted to post the initial patch for feedback)

That needs to be written before it can be installed.

> - The whole "find uses" interface

That can be written later.

> - etags.el integration

That needs to be written before it can be installed.

> - emacs-lisp-mode integration

That can be written later.  It should probably obsolete
find-definition-noselect.

> This also does define a minor mode instead of changing the global key
> bindings.  I think in the final version it should replace the key
> binding definitions done in etags.el.  Is this correct?

Yes, that's correct.

> Do I need to hook it up elsewhere in the build process?

I don't think so.

Thanks, it looks pretty good.  See further comments below.


        Stefan


> +(defcustom find-definition-marker-ring-length 16
> +  "Length of marker rings `find-definition-marker-ring'."
> +  :group 'find-definition
> +  :type 'integer)

The :group is redundant.

> +(defvar find-definition-function nil
> +  "The function `find-definition' calls to find the definition.
> +
> +Will be called with no arguments with point at the location of
> +the thing to find the definition for. It should return a list
> +with each element being a list of one to three elements. The
> +first element should be the file name, the second the
> +line (defaulting to 1) and the third the column (defaulting to
> +0).")

Please use two spaces between sentences.  Also we usually prefer to
describe things using "patterns", as in:

   Will be called with no arguments with point at the location of
   the thing to find the definition for.  It should return a list
   of elements of the form (FILE LINE COL) where LINE and COL can be
   omitted.")

> +(defvar find-definition-identifier-function nil

I suggest you collapse those two function variables into one: if called
with a nil value (or without argument), then just find the definitions
of "thing at point" and if called with a string, then find the
definitions of that identifier.

> +  (let ((map (make-sparse-keymap)))
> +    (define-key map (kbd "M-.") 'find-definition)
> +    (define-key map (kbd "C-x 4 .") 'find-definition-other-window)
> +    (define-key map (kbd "C-x 5 .") 'find-definition-other-frame)
> +    ;; (define-key map (kbd "M-_") 'find-definition-uses)
> +    (define-key map (kbd "M-,") 'find-definition-goto-last-position)

We should probably keep a M-* binding for now.

> +;;;###autoload
> +(define-minor-mode find-definition-mode
> +  "Minor mode to provide some key bindings to find definitions.
> +\\{find-definition-mode-map}"
> +  :keymap 'find-definition-mode)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I don't think this really does what you want.  Better just completely
remove it [ Not that it matters since find-definition-mode should
simply disappear.  ].

> +;;;###autoload
> +(defun find-definition (&optional ask)
> +  "Go to the definition of the thing at point.
> +If the definition can not be found, or with a prefix argument,
> +prompt for a symbol to use."
> +  (interactive "P")
> +  (switch-to-buffer (find-definition--noselect ask)))

Rather than ask from the body of the function, I think the prompting
would be better done in the interactive form.  I.e.

   (defun find-definition (&optional identifier)
     "Go to the definition of the thing at point.
   If the definition can not be found, or with a prefix argument,
   prompt for a symbol to use."
     (interactive
      (if current-prefix-arg (list (find-definition--read-identifier))))
     (switch-to-buffer (find-definition--noselect identifier)))

This of course also suggests we should not do the "if the definition can
not be found, prompt for a symbol to use".

> +      ;; Exactly one definition

Please punctuate your comments.

> +      (with-current-buffer outbuf
> +        (erase-buffer)
> +        (setq default-directory dir)
> +        (compilation-mode)
> +        (dolist (location locations)
> +          (let* ((filename (elt location 0))
> +                 (line (or (elt location 1)
> +                           1))
> +                 (col (or (elt location 2)
> +                          0))
> +                 (buffer (find-buffer-visiting filename))
> +                 (line-string
> +                  (when buffer
> +                    (with-current-buffer buffer
> +                      (save-excursion
> +                        (save-restriction
> +                          (widen)
> +                          (goto-char (point-min))
> +                          (forward-line (- line 1))
> +                          (buffer-substring (line-beginning-position)
> +                                            (line-end-position))))))))
> +            (insert (format "%s:%s:%s:%s\n"
> +                            filename line col
> +                            (or line-string
> +                                "")))))

We can probably use a new mode that derives from compilation-mode.
That will let us use a more efficient and reliable regexp to match the
entries we put in there.
Also, I think that when the LINE&COL are absent, we should not move to
LINE=1, but instead just leave point alone in that buffer (and in the
list, we should not explicitly say "FILE:1:0" but just "FILE:" or
something like that).

One remaining issue is with the filenames themselves: the
find-definition-function may very well (and maybe should usually) return
absolute file names, so we should "prettify" them (make them relative)
before displaying them in the *Definitions* buffer.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> +(defvar find-definition-identifier-function nil
>
> I suggest you collapse those two function variables into one: if called
> with a nil value (or without argument), then just find the definitions
> of "thing at point" and if called with a string, then find the
> definitions of that identifier.

Keeping two variables puts that logic in the front-end, so all the
backends don't have to implement it, and we can change it for everyone
consistently. 

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

>>> +(defvar find-definition-identifier-function nil
>> I suggest you collapse those two function variables into one: if called
>> with a nil value (or without argument), then just find the definitions
>> of "thing at point" and if called with a string, then find the
>> definitions of that identifier.
> Keeping two variables puts that logic in the front-end, so all the
> backends don't have to implement it, and we can change it for everyone
> consistently. 

I think both in the front-end and in many backends, the two cases will
share a lot of code, so using a single var will simplify code in the
front-end and in many backends as well.


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

On Wed, 19 Nov 2014 23:19:28 -0500
Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> >>> +(defvar find-definition-identifier-function nil
> >> I suggest you collapse those two function variables into one: if
> >> called with a nil value (or without argument), then just find the
> >> definitions of "thing at point" and if called with a string, then
> >> find the definitions of that identifier.
> > Keeping two variables puts that logic in the front-end, so all the
> > backends don't have to implement it, and we can change it for
> > everyone consistently. 
> 
> I think both in the front-end and in many backends, the two cases will
> share a lot of code, so using a single var will simplify code in the
> front-end and in many backends as well.

As one concrete example, for Elpy, the two will share absolutely no
code. But the difference is minuscle:

(setq find-definition-function 'elpy-find-definition-by-location)
(setq find-definition-identifier-function 'elpy-find-definition-by-identifier)

vs.

(setq find-definition-function 'elpy-find-definition-by-location-or-identifier)

(defun elpy-find-definition-by-location-or-identifier (&optional identifier)
  (if symbol
      (elpy-find-definition-by-identifier identifier)
    (elpy-find-definition-by-location)))

So I do not really care either way.

Jorgen

Re: Generalizing find-definition

[Helmut Eller]

[-- Attachment #1: Type: text/plain, Size: 524 bytes --]

On Mon, Nov 17 2014, Jorgen Schaefer wrote:

> Some initial patch attached.

I'm proposing an alternative implementation.  This version uses EIOIO
which, I think, makes it more flexible.  Proof-of-concept-quality
backends for elisp and etags are included.  The UI is lifted from SLIME
(with some simplifications).  To try it out, load xref.el and then M-x
xref-minor-mode to turn it on.  It's not fully polished but good enough
to show the main ideas.

Also available at: https://github.com/ellerh/xref/blob/master/xref.el


[-- Attachment #2: xref.el --]
[-- Type: application/emacs-lisp, Size: 15519 bytes --]

[-- Attachment #3: Type: text/plain, Size: 8 bytes --]


Helmut

Re: Generalizing find-definition

[Jorgen Schaefer]

On Thu, 20 Nov 2014 14:44:54 +0100
Helmut Eller <eller.helmut@gmail.com> wrote:

> On Mon, Nov 17 2014, Jorgen Schaefer wrote:
> 
> > Some initial patch attached.
> 
> I'm proposing an alternative implementation.

Fine with me, too, as long as I have a trivial way of providing common
functionality. (M-. is by far not the only place.)

I did not see any tests in the repo, did I look in the wrong spot?

Before I continue to put work into this, Stefan, which way do you
prefer - functions or EIEIO?

Jorgen

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Nov 20 2014, Jorgen Schaefer wrote:

> I did not see any tests in the repo, did I look in the wrong spot?

No, there are no tests.  To try things out use M-x xref-minor-mode and
then press M-. on a symbol.  I only tried it with emacs-lisp-mode and
the C code of Emacs.

Helmut

Re: Generalizing find-definition

[Stefan Monnier]

> Before I continue to put work into this, Stefan, which way do you
> prefer - functions or EIEIO?

Either way is fine by me,


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

On Thu, 20 Nov 2014 18:27:08 -0500
Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> > Before I continue to put work into this, Stefan, which way do you
> > prefer - functions or EIEIO?
> 
> Either way is fine by me,

I am afraid someone will have to pick one, because we can hardly
include both in Emacs. And I am not sure who would be able to pick one
if not you.

How would you like us to decide which one to include?

(Pistols at dawn might be overdoing it!)

Jorgen

Re: Generalizing find-definition

[Stefan Monnier]

> (Pistols at dawn might be overdoing it!)

Just remember to record it in a Free format,


        Stefan

Re: Generalizing find-definition

[martin rudalics]

> I am afraid someone will have to pick one, because we can hardly
> include both in Emacs.

Why not?  If you're interested I can send you mine too ;-)

martin

Re: Generalizing find-definition

[Stefan Monnier]

>> I am afraid someone will have to pick one, because we can hardly
>> include both in Emacs.
> Why not?  If you're interested I can send you mine too ;-)

We can include them all.  But we can only bind M-. to one of them in the
default configuration.  And I don't think we want to ask major modes to
provide more than one backend.


        Stefan

Re: Generalizing find-definition

[Johan Claesson]

Awesome library.  For people just as lazy as me i recommend the
following keybindings to further reduce finger movement:

(define-key xref--xref-buffer-mode-map [?,] 'xref-prev-line)
(define-key xref--xref-buffer-mode-map [?.] 'xref-next-line)

Regards,

/Johan



Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Nov 17 2014, Jorgen Schaefer wrote:
>
>> Some initial patch attached.
>
> I'm proposing an alternative implementation.  This version uses EIOIO
> which, I think, makes it more flexible.  Proof-of-concept-quality
> backends for elisp and etags are included.  The UI is lifted from SLIME
> (with some simplifications).  To try it out, load xref.el and then M-x
> xref-minor-mode to turn it on.  It's not fully polished but good enough
> to show the main ideas.
>
> Also available at: https://github.com/ellerh/xref/blob/master/xref.el
>
>
>
>
> Helmut

Re: Generalizing find-definition

[Helmut Eller]

On Sun, Nov 23 2014, Johan Claesson wrote:

> Awesome library.  For people just as lazy as me i recommend the
> following keybindings to further reduce finger movement:
>
> (define-key xref--xref-buffer-mode-map [?,] 'xref-prev-line)
> (define-key xref--xref-buffer-mode-map [?.] 'xref-next-line)

Perhaps a bit exotic, but I guess it doesn't hurt to have them.
I added it to the code at: https://github.com/ellerh/xref.

I also changed the interface to find the backend: the buffer-local
variable xref-backend is replaced by xref-backend-function -- a function
that should return the backend.  This indirection makes it easier to
autoload xref.el.

Helmut

Re: Generalizing find-definition

[Stephen Leake]

Using the code at: https://github.com/ellerh/xref, I've written an
ada-mode backend (not completion yet). It was quite straight-forward,
and I like the dispatching that eieio provides.

So I recommend we merge this code into emacs master.

A couple comments:

The current copyright on xref.el is Helmut Eller; I assume you've got a
copyright assignment on file.

I'd like to add to xref.el:

(defun xref-find-definition-at-point ()
  (interactive)
  (xref--find-definition (xref-identifier-at-point (xref--backend)) nil))

That's the function I use most often.


There is a FIXME on xref-push-marker-stack. I gather you'd like this to
be independent of etags?

It would not be hard to implement an independent marker ring/stack. But
I think it makes more sense to use the etags marker ring; that way,
if I am navigating thru code that uses multiple languages, and one
language mode uses xref while another uses tags, there is still only one
tag ring.

Eventually, when most modes have migrated to xref, it might make sense
to switch to a separate marker ring.

-- 
-- Stephe

Re: Generalizing find-definition

[Stephen Leake]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Using the code at: https://github.com/ellerh/xref, I've written an
> ada-mode backend (not completion yet). It was quite straight-forward,
> and I like the dispatching that eieio provides.
>
> So I recommend we merge this code into emacs master.
>
> A couple comments:

I used xref--file-location, which is an internal symbol:

(defmethod xref-find-definitions ((_ xref-ada-backend) identifier)
    ...
    (list
     (xref-make
     ""
     (make-instance
      'xref--file-location
      :file (nth 0 target)
      :line (nth 1 target)
      :column (nth 2 target))))
    ))

There is a non-internal alternative xref-file-location:

(defmethod xref-find-definitions ((_ xref-ada-backend) identifier)
    ...
    (list
     (xref-make
     ""
     (xref-file-location
      (nth 0 target)
      (nth 1 target)
      (nth 2 target)))
    ))

I find the first more readable; that's part of the point of using the
class paradigm.

So I suggest promoting the classes in xref.el to a supported part of the
API.

In addition, that allows the code in '...' to change to also use the
xref--file-location class, which would be a Good Thing.

-- 
-- Stephe

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 04 2014, Stephen Leake wrote:

[...]
> I used xref--file-location, which is an internal symbol:
[...]
> There is a non-internal alternative xref-file-location:
[...]

My intention for that was to export constructors but keep the
representation private.

> So I suggest promoting the classes in xref.el to a supported part of the
> API.

OK. I have export the class symbols now and renamed the constructors
to xref-make-FOO.

Helmut

Re: Generalizing find-definition

[Helmut Eller]

On Wed, Dec 03 2014, Stephen Leake wrote:

[...]
> A couple comments:
>
> The current copyright on xref.el is Helmut Eller; I assume you've got a
> copyright assignment on file.

Yes, I have the paperwork.

> I'd like to add to xref.el:
>
> (defun xref-find-definition-at-point ()
>   (interactive)
>   (xref--find-definition (xref-identifier-at-point (xref--backend)) nil))
>
> That's the function I use most often.

Hmm, xref-find-definition does this, except for the case when it's not
called interactively.  Do you need the non-interactive version?  Maybe
it would be better to make xref-find-definition more useful
non-interactively than to define an almost identical function.

> There is a FIXME on xref-push-marker-stack. I gather you'd like this to
> be independent of etags?
>
> It would not be hard to implement an independent marker ring/stack. But
> I think it makes more sense to use the etags marker ring; that way,
> if I am navigating thru code that uses multiple languages, and one
> language mode uses xref while another uses tags, there is still only one
> tag ring.
>
> Eventually, when most modes have migrated to xref, it might make sense
> to switch to a separate marker ring.

My idea was to move the ring from etags.el to xref.el and perhaps define
some aliases for backward compatibility.

Helmut

Re: Generalizing find-definition

[Stephen Leake]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Wed, Dec 03 2014, Stephen Leake wrote:
>
> [...]
>> A couple comments:
>>
>> The current copyright on xref.el is Helmut Eller; I assume you've got a
>> copyright assignment on file.
>
> Yes, I have the paperwork.
>
>> I'd like to add to xref.el:
>>
>> (defun xref-find-definition-at-point ()
>>   (interactive)
>>   (xref--find-definition (xref-identifier-at-point (xref--backend)) nil))
>>
>> That's the function I use most often.
>
> Hmm, xref-find-definition does this, except for the case when it's not
> called interactively.  

I bound xref-find-definition to a key. When I invoke that key, I'm
prompted for the identifier; the default is the identifier at point, but
I still have to hit enter. I want to eliminate that prompt/enter step.

Currently in ada-mode, C-c C-d gives the non-prompt identifier at point
behavior; that makes it very fluid to drill down thru several layers of
calls.

> Do you need the non-interactive version? Maybe
> it would be better to make xref-find-definition more useful
> non-interactively than to define an almost identical function.

I think xref-find-definition is fine as is; you can call it
interactively to be prompted with completion, or programmatically and
pass it an identifier. But that misses the interactive non-prompt case.
We could use C-u or some other prefix to control that, but a separate
function is easier to bind to a key.

>> There is a FIXME on xref-push-marker-stack. I gather you'd like this to
>> be independent of etags?
>>
>> It would not be hard to implement an independent marker ring/stack. But
>> I think it makes more sense to use the etags marker ring; that way,
>> if I am navigating thru code that uses multiple languages, and one
>> language mode uses xref while another uses tags, there is still only one
>> tag ring.
>>
>> Eventually, when most modes have migrated to xref, it might make sense
>> to switch to a separate marker ring.
>
> My idea was to move the ring from etags.el to xref.el and perhaps define
> some aliases for backward compatibility.

+1

-- 
-- Stephe

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 04 2014, Stephen Leake wrote:

>> Hmm, xref-find-definition does this, except for the case when it's not
>> called interactively.  
>
> I bound xref-find-definition to a key. When I invoke that key, I'm
> prompted for the identifier; the default is the identifier at point, but
> I still have to hit enter.

Something's odd here.  xref-find-definition does NOT prompt by default.
E.g. if you invoke M-x xref-find-definition in an elisp buffer with the
text "cons" around point then it should take you immediately to alloc.c
without any prompting.  xref-find-definition should prompt only if
either xref-identifier-at-point returns nil or if it's invoked with
prefix argument.

> I want to eliminate that prompt/enter step.

So do I.

Helmut

Re: Generalizing find-definition

[Stephen Leake]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Thu, Dec 04 2014, Stephen Leake wrote:
>
>>> Hmm, xref-find-definition does this, except for the case when it's not
>>> called interactively.  
>>
>> I bound xref-find-definition to a key. When I invoke that key, I'm
>> prompted for the identifier; the default is the identifier at point, but
>> I still have to hit enter.
>
> Something's odd here.  xref-find-definition does NOT prompt by
> default.

Ah. It only prompts if xref-identifier-at-point returns nil; I must have
tested find-definition before I finished implementing
identifier-at-point.

Sorry for the noise.


Other comments:

xref.el needs (provide 'xref).


For elisp, when I have a variable and function with the same name, the
*xref* buffer shows:

c:/Projects/org.emacs.ada-mode/wisi.el
  wisi-number-p
  (defvar wisi-number-p)

That would be more consistent if the first reference was shown as:

  (defun wisi-number-p)


In ada-mode, I use compilation-mode for showing multiple references;
that is a familiar UI. I gather the mode you implemented is similar to
SLIME? Perhaps we need another dispatch/user option to choose this UI?


For Ada, only "find uses of identifier" returns multiple locations. That
function is not in xref.el yet. I suggest:

(defun xref-find-references (&optional identifier)
  (interactive (list (xref--read-identifier "Find references of: ")))
  (xref--find-references identifier nil))

similar to find-definitions.

Hmm, the dispatching name has to be different; perhaps
xref-find-references-m (for 'method')?

I'd actually prefer xref-find-definitions as the user function, and
xref-find-defintions-m as the dispatching function; the user function
can show more than one definition.

-- 
-- Stephe

Re: Generalizing find-definition

[Helmut Eller]

On Fri, Dec 05 2014, Stephen Leake wrote:

> Other comments:
>
> xref.el needs (provide 'xref).

Added.

[...]
> That would be more consistent if the first reference was shown as:
>
>   (defun wisi-number-p)

OK.

> In ada-mode, I use compilation-mode for showing multiple references;
> that is a familiar UI. I gather the mode you implemented is similar to
> SLIME?

Yes, it's a simplified version of SLIME's UI.

> Perhaps we need another dispatch/user option to choose this UI?

I added a variable xref-show-xrefs-function so that people can
experiment with alternative UIs.  I'm not sure if compilation-mode is
able to handle locations that can't be represented as simple strings.

[...]
> I'd actually prefer xref-find-definitions as the user function, and
> xref-find-defintions-m as the dispatching function; the user function
> can show more than one definition.

I decided to call the backend functions xref-lookup-defintions and
xref-lookup-references and the commands xref-find-defintions
resp. xref-find-references.

Helmut

Re: Generalizing find-definition

[Stephen Leake]

Helmut Eller <eller.helmut@gmail.com> writes:

>> Perhaps we need another dispatch/user option to choose this UI?
>
> I added a variable xref-show-xrefs-function so that people can
> experiment with alternative UIs.  I'm not sure if compilation-mode is
> able to handle locations that can't be represented as simple strings.

Ok, I'll give that a try.

> [...]
>> I'd actually prefer xref-find-definitions as the user function, and
>> xref-find-defintions-m as the dispatching function; the user function
>> can show more than one definition.
>
> I decided to call the backend functions xref-lookup-defintions and
> xref-lookup-references and the commands xref-find-defintions
> resp. xref-find-references.

Ok.


Another aspect of "cross reference" is to follow links such as:

1) http://www.gnu.org/software/emacs

2) admin/notes/commits

3) (info "(elisp)Syntax Class Table" "*info syntax class table*")

Currently, 1) is handled by browse-url-at-point, which is not bound to
any key by default.

Similarly, 2) is find-file-at-point

3) is handled by C-x C-e; I don't think we need to change that.

I have a function that combines 1 and 2 and similar links; it calls
ffap-string-at-point, compares that to an alist of (regexp . command),
and then defaults to find-file. The alist has:

      (cons "^ftp://" 'browse-url-at-point)
      (cons "^http://" 'browse-url-at-point)
      (cons "^https://" 'browse-url-at-point)
      (cons "\\.bmp$" 'sal-w32-open)
      (cons "\\.bz2$" 'sal-w32-open)

etc.

Perhaps a similar function could be included in xref?

-- 
-- Stephe

Re: Generalizing find-definition

[Helmut Eller]

On Fri, Dec 05 2014, Stephen Leake wrote:

[...]
> I have a function that combines 1 and 2 and similar links; it calls
> ffap-string-at-point, compares that to an alist of (regexp . command),
> and then defaults to find-file. The alist has:
[...]
> Perhaps a similar function could be included in xref?

Just thinking aloud:

I guess such filenames or URLs (URIs?) typically occur in comments,
string literals, or perhaps in things like #include <stdio.h> or
(require 'some-library).  M-x ffap seems cover this partially, but then
there's also M-x find-library.  Certainly a lot of things to find!

A problem seems to be that xref-identifier-at-point would need a
possibly complicated heuristic to determine if we are at such a filename
or a "normal" identifier.  Maybe it's easier to have a separate
xref-file-name-at-point which would by default do what
ffap-guess-file-name-at-point does.  For ELisp it should additionally
recognize (require 'foo) and somehow reuse find-library.

There's also the problem which keybinding to use.  We will probably
steal the global bindings for M-. and M-, from etags but beyond that
we don't have keys for our commands.  At least one additional prefix key
for a xref keymap would be nice.

Helmut

Re: Generalizing find-definition

[Stephen Leake]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Fri, Dec 05 2014, Stephen Leake wrote:
>
> [...]
>> I have a function that combines 1 and 2 and similar links; it calls
>> ffap-string-at-point, compares that to an alist of (regexp . command),
>> and then defaults to find-file. The alist has:
> [...]
>> Perhaps a similar function could be included in xref?
>
> Just thinking aloud:
>
> I guess such filenames or URLs (URIs?) typically occur in comments,
> string literals, or perhaps in things like #include <stdio.h> or
> (require 'some-library).  M-x ffap seems cover this partially, but then
> there's also M-x find-library.  Certainly a lot of things to find!

I just added this for (require 'foo):

  (set (make-local-variable 'ff-search-directories) 'load-path)
  (set (make-local-variable 'ff-special-constructs) nil)
  (add-to-list
   'ff-special-constructs
   (cons "^(require '\\(.*\\))" (lambda () (file-name-nondirectory (locate-library (match-string 1))))))

That's used by ff-find-other-file.

> A problem seems to be that xref-identifier-at-point would need a
> possibly complicated heuristic to determine if we are at such a filename
> or a "normal" identifier.  

Yes, I think that requires input from the user. Currently, I use C-F11
(bound to 'ff-find-other-file') when the thing at point is something
related to a filename ('require' in elisp, '#include' in C++, 'with' in
Ada), and C-c C-d (now bound to 'xref-find-definitions') when the thing
at point is a symbol.

We could just search for both, and present a list, but I think that
would be more cumbersome than it's worth.

> Maybe it's easier to have a separate xref-file-name-at-point which
> would by default do what ffap-guess-file-name-at-point does. For ELisp
> it should additionally recognize (require 'foo) and somehow reuse
> find-library.

find-file.el and files.el provide similarly customizable 'find a file'
functions. It would be nice to unify them, but that's a lot of work.

I'm most familiar with find-file, but it could easily be that files is a
better structure.

> There's also the problem which keybinding to use.  We will probably
> steal the global bindings for M-. and M-, from etags but beyond that
> we don't have keys for our commands.  At least one additional prefix key
> for a xref keymap would be nice.

ada-mode uses C-c C-o for a wrapper around ff-find-other-file. I don't
see a similar binding in c-mode.

We would probably need to keep xref-minor-mode for now, and provide a
binding in there. Then if it becomes popular, it can be promoted to a
global binding.

-- 
-- Stephe

RE: Generalizing find-definition

[Drew Adams]

> A problem seems to be that xref-identifier-at-point would need a
> possibly complicated heuristic to determine if we are at such a
> filename or a "normal" identifier. Maybe it's easier to have a
> separate xref-file-name-at-point which would by default do what
> ffap-guess-file-name-at-point does.  For ELisp it should
> additionally recognize (require 'foo) and somehow reuse find-library.

And..., and somehow..., and..., and somehow...

In general, this is the wrong approach, I think.

Q. When does reuse of a file name or a URL or... at point make
   sense?
A. As input to a command that acts on a file name or a URL or...

The *particular command* already knows what kind of thing it
expects.  All that's needed, as a *general* mechanism, is a
general way to pull text at point into the minibuffer.

That's where the design effort should be, IMO.  And this
mechanism should be entirely decoupled from any particular
use of the grabbed text.  That's my main point here.

In particular, `find-definition' itself might be misguided.
I cannot really speak to whether it is, but that's my hunch.

Doesn't a user know what kind of thing definition s?he wants
to look up?  Does s?he really need Emacs to guess what kind
of thing is at point (a guess that is fraught with ambiguity)?

The problem comes down, I think, to providing a key (or keys)
that will *grab what the user wants at point*.

We should then bind that key in `minibuffer-local-map' so that
it is available *always*, regardless of the minibuffer-reading
command and the particular way of reading (with completion or
not, and for any kind of completion).

There is room for imagination and invention here, and that is
where the effort should be placed, IMHO.

Let's suppose that we start by trying for just a single key -
`M-.', for instance.  Clearly, adding more "grab" keys would
make it easier for a user to make known to Emacs what to grab,
but it would also make users know more keys etc.  We can always
add more.  Let's see what can be done with one, for starters.

As food for thought, here is a description of what `M-.' does
in Icicles, from the minibuffer.  I don't claim that this is
the full solution - at all.  As I said, I think there is plenty
of room for invention, to come up with good ways for a user to
tell Emacs what thing(s) to grab at point.

Icicles currently uses a single key, `M-.', to grab stuff into
the minibuffer from point.  You customize what your general
preferences are in this regard.

The choice you make is whether repeated use of `M-.' should,
by default, grab (a) additional stuff of the same type or
(b) alternative stuff of different types (i.e., replacing,
in the minibuffer, what the previous invocation grabbed).

You can override your general preference (alternatives vs
more-of-the-same) on the fly when you use `M-.', by
providing a prefix arg.

For example, if you generally prefer that repeated `M-.'
grab different kinds of thing at point (default behavior),
then `C-u M-.' switches the behavior temporarily so that
repeating `M-.' grabs successive occurrences of the same
kind of thing.

There is additional, more complex behavior available as
well.  You can customize the sequence of functions used to
grab different kinds of thing.  You can grab successive
things of the same type to the left or to the right of
point.  You can grab alternative things at point and
insert not the grabbed text but the result of evaluating
it as a Lisp sexp.

This page describes the behavior of `M-.' in Icicles:
http://www.emacswiki.org/Icicles_-_Inserting_Text_from_Cursor
And this part describes the details outlined above:
http://www.emacswiki.org/Icicles_-_Inserting_Text_from_Cursor#RepeatedM.

To be clear, I am not at all proposing this same behavior
for Emacs.  I point to it as food for thought.

What I am suggesting is only this:

1. It makes sense to concentrate on coming up with a
   useful, general-purpose grab-text-at-point minibuffer
   command, not with a general-purpose `find-definition'
   top-level command.

2. What Icicles does in this regard might serve as some
   food for thought.  But there are surely more and
   better possibilities.

3. Everything useful need not be combined in a single
   minibuffer command (such as Icicles tries to do with
   `M-.').  Having different, easily understood behaviors
   on separate keys can also be a good way to go.

HTH.

Re: Generalizing find-definition

[Stephen Leake]

Drew Adams <drew.adams@oracle.com> writes:

>> A problem seems to be that xref-identifier-at-point would need a
>> possibly complicated heuristic to determine if we are at such a
>> filename or a "normal" identifier. Maybe it's easier to have a
>> separate xref-file-name-at-point which would by default do what
>> ffap-guess-file-name-at-point does.  For ELisp it should
>> additionally recognize (require 'foo) and somehow reuse find-library.
>
> And..., and somehow..., and..., and somehow...
>
> In general, this is the wrong approach, I think.
>
> Q. When does reuse of a file name or a URL or... at point make
>    sense?
> A. As input to a command that acts on a file name or a URL or...
>
> The *particular command* already knows what kind of thing it
> expects.  

Not in my use case.

I'd like to have one "goto ref" key that Does the Right Thing as much as
possible. That way I don't have to remember different keys for different
cases. 

> Doesn't a user know what kind of thing definition s?he wants
> to look up?  

Yes, but I don't want to _also_ have to know which key to invoke.

> Does s?he really need Emacs to guess what kind of thing is at point 

Not "guess", no. Determine precisely, yes.

> (a guess that is fraught with ambiguity)?

If Emacs can't figure it out, put up a list of possibilities, or report
error.

This works very well for me. I have three related keys for this kind of
thing (Emacs isn't quite as intelligent as I'd like yet);

C-c C-d for symbols, whose behavior is mostly determined by the
compiler generated cross reference info; now bound to
'xref-find-definitions'. 

C-F11 for files, whose behavior is determined by parsing the string at
point, influenced by the major mode. Mostly a wrapper around
ff-find-the-other-file.

C-F12 whose behavior is totally determined by parsing the string at
point or active region, via an alisp of regexps. Sometimes the
"at-point" version does the wrong thing; then the "active region"
version does what I actually want (and vice versa).

I started the current discussion (I changed the Subject line) to cover
implementing the C-F12 functionality in a more general way, or just make
it available in Emacs core or an ELPA package.

It might also make sense to make the C-F11 functionality available in
Emacs core.

> The problem comes down, I think, to providing a key (or keys)
> that will *grab what the user wants at point*.

Thing-at-point does that pretty well, and xref-identifier-at-point
provides a way to extend that.

I'm talking about what to do with the string after we get it, and to
some extent UI design to give more information to Emacs about what to
get and what to do with it (at point vs region, for example).

> We should then bind that key in `minibuffer-local-map' so that
> it is available *always*, regardless of the minibuffer-reading
> command and the particular way of reading (with completion or
> not, and for any kind of completion).

This might make sense, but that requires being in a prompt in the first
place; I try to avoid that whenever possible.

> 1. It makes sense to concentrate on coming up with a
>    useful, general-purpose grab-text-at-point minibuffer
>    command, not with a general-purpose `find-definition'
>    top-level command.

We disagree; that's fine.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> I guess such filenames or URLs (URIs?) typically occur in comments,
> string literals, or perhaps in things like #include <stdio.h> or
> (require 'some-library).  M-x ffap seems cover this partially, but then
> there's also M-x find-library.  Certainly a lot of things to find!

Indeed, and I don't think M-. should care about those cases.
C-x C-f is supposed to handle those cases instead.

This said, I think the most urgent step is to get such a generalized
find-definition into Emacs.


        Stefan

Re: Generalizing find-definition

[Helmut Eller]

[-- Attachment #1: Type: text/plain, Size: 164 bytes --]

On Sat, Dec 06 2014, Stefan Monnier wrote:

> This said, I think the most urgent step is to get such a generalized
> find-definition into Emacs.

Here is a patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Generalized-infrastructure-for-find-definition.patch --]
[-- Type: text/x-diff, Size: 26291 bytes --]

From 001c23b21450001667adaa27d0b27061a4c26c3c Mon Sep 17 00:00:00 2001
From: Helmut Eller <eller.helmut@gmail.com>
Date: Sun, 7 Dec 2014 10:46:33 +0100
Subject: [PATCH] Generalized infrastructure for find-definition

* progmodes/xref.el: New file.

* progmodes/etags.el (find-tag-marker-ring, pop-tag-mark): Move to
xref but keep aliases for backward compatibility.
(tags-reset-tags-tables): Use xref marker stack instead of
find-tag-marker-ring.
(etags--xref-backend, etags--xref-backend-var)
(etags-xref-backend-function): New xref backend.
(esc-map, ctl-x-4-map, ctl-x-5-map): Move key bindings for M-.,
M-, C-x 4 M-., and C-x 5 M-. to xref.el

* progmodes/elisp-mode.el (emacs-lisp-mode): Initialize
xref-backend-function.
---
 lisp/ChangeLog               |   18 ++
 lisp/progmodes/elisp-mode.el |    1 +
 lisp/progmodes/etags.el      |   80 +++++--
 lisp/progmodes/xref.el       |  491 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 567 insertions(+), 23 deletions(-)
 create mode 100644 lisp/progmodes/xref.el

diff --git a/lisp/ChangeLog b/lisp/ChangeLog
index b3cb2fa..46baac5 100644
--- a/lisp/ChangeLog
+++ b/lisp/ChangeLog
@@ -1,3 +1,21 @@
+2014-12-07  Helmut Eller  <eller.helmut@gmail.com>
+
+	Generalized infrastructure for find-definition
+
+	* progmodes/xref.el: New file.
+
+	* progmodes/etags.el (find-tag-marker-ring, pop-tag-mark): Move to
+	xref but keep aliases for backward compatibility.
+	(tags-reset-tags-tables): Use xref marker stack instead of
+	find-tag-marker-ring.
+	(etags--xref-backend, etags--xref-backend-var)
+	(etags-xref-backend-function): New xref backend.
+	(esc-map, ctl-x-4-map, ctl-x-5-map): Move key bindings for M-.,
+	M-, C-x 4 M-., and C-x 5 M-. to xref.el
+
+	* progmodes/elisp-mode.el (emacs-lisp-mode): Initialize
+	xref-backend-function.
+
 2014-12-05  Juri Linkov  <juri@linkov.net>
 
 	* comint.el (comint-history-isearch-search)
diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ba70f90..2e68516 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -231,6 +231,7 @@ Blank lines separate paragraphs.  Semicolons start comments.
   (setq imenu-case-fold-search nil)
   (setq-local eldoc-documentation-function
               #'elisp-eldoc-documentation-function)
+  (setq-local xref-backend-function 'xref-elisp-backend-function)
   (add-hook 'completion-at-point-functions
             #'elisp-completion-at-point nil 'local))
 
diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index b89b4cf..c1c6303 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -28,6 +28,7 @@
 
 (require 'ring)
 (require 'button)
+(require 'xref)
 
 ;;;###autoload
 (defvar tags-file-name nil
@@ -182,8 +183,8 @@ Example value:
 		       (sexp :tag "Tags to search")))
   :version "21.1")
 
-(defvar find-tag-marker-ring (make-ring find-tag-marker-ring-length)
-  "Ring of markers which are locations from which \\[find-tag] was invoked.")
+(defvaralias 'find-tag-marker-ring 'xref--marker-ring)
+(make-obsolete-variable 'find-tag-marker-ring nil "25.1")
 
 (defvar default-tags-table-function nil
   "If non-nil, a function to choose a default tags file for a buffer.
@@ -716,12 +717,10 @@ Returns t if it visits a tags table, or nil if there are no more in the list."
     (while (< i find-tag-marker-ring-length)
       (if (aref (cddr tags-location-ring) i)
 	  (set-marker (aref (cddr tags-location-ring) i) nil))
-      (if (aref (cddr find-tag-marker-ring) i)
-	  (set-marker (aref (cddr find-tag-marker-ring) i) nil))
       (setq i (1+ i))))
+  (xref-clear-marker-stack)
   (setq tags-file-name nil
 	tags-location-ring (make-ring find-tag-marker-ring-length)
-	find-tag-marker-ring (make-ring find-tag-marker-ring-length)
 	tags-table-list nil
 	tags-table-computed-list nil
 	tags-table-computed-list-for nil
@@ -898,7 +897,7 @@ See documentation of variable `tags-file-name'."
 	      ;; Run the user's hook.  Do we really want to do this for pop?
 	      (run-hooks 'local-find-tag-hook))))
       ;; Record whence we came.
-      (ring-insert find-tag-marker-ring (point-marker))
+      (xref-push-marker-stack)
       (if (and next-p last-tag)
 	  ;; Find the same table we last used.
 	  (visit-tags-table-buffer 'same)
@@ -954,7 +953,6 @@ See documentation of variable `tags-file-name'."
 	(switch-to-buffer buf)
       (error (pop-to-buffer buf)))
     (goto-char pos)))
-;;;###autoload (define-key esc-map "." 'find-tag)
 
 ;;;###autoload
 (defun find-tag-other-window (tagname &optional next-p regexp-p)
@@ -995,7 +993,6 @@ See documentation of variable `tags-file-name'."
 			;; the window's point from the buffer.
 			(set-window-point (selected-window) tagpoint))
 		      window-point)))
-;;;###autoload (define-key ctl-x-4-map "." 'find-tag-other-window)
 
 ;;;###autoload
 (defun find-tag-other-frame (tagname &optional next-p)
@@ -1020,7 +1017,6 @@ See documentation of variable `tags-file-name'."
   (interactive (find-tag-interactive "Find tag other frame: "))
   (let ((pop-up-frames t))
     (find-tag-other-window tagname next-p)))
-;;;###autoload (define-key ctl-x-5-map "." 'find-tag-other-frame)
 
 ;;;###autoload
 (defun find-tag-regexp (regexp &optional next-p other-window)
@@ -1049,20 +1045,8 @@ See documentation of variable `tags-file-name'."
 ;;;###autoload (define-key esc-map "*" 'pop-tag-mark)
 
 ;;;###autoload
-(defun pop-tag-mark ()
-  "Pop back to where \\[find-tag] was last invoked.
+(defalias 'pop-tag-mark 'xref-pop-marker-stack)
 
-This is distinct from invoking \\[find-tag] with a negative argument
-since that pops a stack of markers at which tags were found, not from
-where they were found."
-  (interactive)
-  (if (ring-empty-p find-tag-marker-ring)
-      (error "No previous locations for find-tag invocation"))
-  (let ((marker (ring-remove find-tag-marker-ring 0)))
-    (switch-to-buffer (or (marker-buffer marker)
-                          (error "The marked buffer has been deleted")))
-    (goto-char (marker-position marker))
-    (set-marker marker nil nil)))
 \f
 (defvar tag-lines-already-matched nil
   "Matches remembered between calls.") ; Doc string: calls to what?
@@ -1859,7 +1843,6 @@ nil, we exit; otherwise we scan the next file."
     (and messaged
 	 (null tags-loop-operate)
 	 (message "Scanning file %s...found" buffer-file-name))))
-;;;###autoload (define-key esc-map "," 'tags-loop-continue)
 
 ;;;###autoload
 (defun tags-search (regexp &optional file-list-form)
@@ -2077,6 +2060,57 @@ for \\[find-tag] (which see)."
       (completion-in-region (car comp-data) (cadr comp-data)
 			    (nth 2 comp-data)
 			    (plist-get (nthcdr 3 comp-data) :predicate)))))
+
+\f
+;;; Xref backed
+
+(defclass etags--xref-backend (xref-backend-class) ())
+
+(defvar etags--xref-backend-var (make-instance 'etags--xref-backend))
+
+;;;###autoload
+(defun etags-xref-backend-function () etags--xref-backend-var)
+
+(defmethod xref-lookup-definitions ((_ etags--xref-backend) id)
+  ;; This emulates the behaviour of `find-tag-in-order' but instead of
+  ;; returning one match at a time all matches are returned as list.
+  ;; NOTE: find-tag-tag-order is typically a buffer-local variable.
+  (let* ((xrefs '())
+	 (first-time t)
+	 (regexp? (consp id))
+	 (pattern (if regexp? (cadr id) id))
+	 (search-fun (if regexp? #'re-search-forward #'search-forward))
+	 (marks (make-hash-table :test 'equal)))
+    (save-excursion
+      (while (visit-tags-table-buffer (not first-time))
+	(setq first-time nil)
+	(dolist (order-fun (cond (regexp? find-tag-regexp-tag-order)
+				 (t find-tag-tag-order)))
+	  (goto-char (point-min))
+	  (while (funcall search-fun pattern nil t)
+	    (when (funcall order-fun pattern)
+	      (beginning-of-line)
+	      (cl-destructuring-bind (hint line &rest pos) (etags-snarf-tag)
+		(unless (eq hint t) ; hint==t if we are in a filename line
+		  (let* ((file (file-of-tag))
+			 (mark-key (cons file line)))
+		    (unless (gethash mark-key marks)
+		      (let ((loc (xref-make-file-location
+				  (expand-file-name file) line 0)))
+			(push (xref-make hint loc) xrefs)
+			(puthash mark-key t marks)))))))))))
+    (nreverse xrefs)))
+
+;; If the text in the minibuffer starts with " it's interpreted as a
+;; regexp.  This is an example for a non-trivial identifier type.
+(defmethod xref-read-identifier-from-minibuffer ((_ etags--xref-backend)
+						 prompt init)
+  (let ((string (read-from-minibuffer prompt init)))
+    (cond ((string-match "^\"" string)
+	   `(rx ,(read string)))
+	  (t
+	   string))))
+
 \f
 (provide 'etags)
 
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
new file mode 100644
index 0000000..911b3bc
--- /dev/null
+++ b/lisp/progmodes/xref.el
@@ -0,0 +1,491 @@
+;; xref.el --- Cross referencing commands              -*-lexical-binding:t-*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; Author: Helmut Eller <eller.helmut@gmail.com>
+
+;; This work is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This work is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; This file provides a somewhat generic infrastructure for cross
+;; referencing commands, in particular "find-definition".  Some part of
+;; the functionality must be implemented in a language dependent way
+;; and that's done by defining a "backend".  The generic code finds
+;; the backend by calling the function stored in the variable
+;; `xref-backend-function'.  A language specific mode usually makes
+;; `xref-backend-function' buffer local before storing into it.
+;;
+;; A backend is an instance of the EIEIO class xref-backend-class.
+;; Various generic functions (in the EIEIO sense of the word) are
+;; defined on xref-backend-class.  A language specific mode usually
+;; creates a subclasses of xref-backend-class and provides specialized
+;; methods for the generic functions.  See the `xref--elisp-backend'
+;; and `etags--xref-backend' classes for examples.
+
+(require 'cl-lib)
+(require 'eieio)
+(require 'ring)
+(require 'find-func) ; for elisp backend
+
+\f
+;;; Locations
+
+;; A location represents a position in a file or buffer.
+(defclass xref-location () ())
+
+;; If a backend decides to subclass xref-location it can provide
+;; methods for some of the following functions:
+(defgeneric xref-location-buffer (location))
+(defgeneric xref-location-position (location))
+(defgeneric xref-location= (location1 location2))
+(defmethod xref-location= ((l1 xref-location) l2)
+  (equal l1 l2))
+
+;;;; Commonly needed location classes are defined here:
+
+;; A file location is a file/line/colum triple.  Line numbers start
+;; from 1 and columns from 0 (as inconstistent as the rest of Emacs).
+;;
+;; FIXME: might be useful to have an optional "hint" i.e. a string to
+;; search for in case the line number is sightly out of date.
+(defclass xref-file-location (xref-location)
+  ((file :type string :initarg :file)
+   (line :type fixnum :initarg :line)
+   (column :type fixnum :initarg :column)))
+
+(defun xref-make-file-location (file line column)
+  (make-instance 'xref-file-location :file file :line line :column column))
+
+(defmethod xref-location-buffer ((l xref-file-location))
+  (with-slots (file) l
+    (or (get-file-buffer file)
+	(let ((find-file-suppress-same-file-warnings t))
+	  (find-file-noselect file)))))
+
+(defmethod xref-location-position ((l xref-file-location))
+  (with-slots (line column) l
+    (with-current-buffer (xref-location-buffer l)
+      (save-restriction
+	(widen)
+	(save-excursion
+	  (goto-char (point-min))
+	  (beginning-of-line line)
+	  (move-to-column column)
+	  (point))))))
+
+(defclass xref-buffer-location (xref-location)
+  ((buffer :type buffer :initarg :buffer :reader xref-location-buffer)
+   (position :type fixnum :initarg :position :reader xref-location-position)))
+
+(defun xref-make-buffer-location (buffer position)
+  (make-instance 'xref-buffer-location :buffer buffer :position position))
+
+;; "Bogus" locations are sometimes useful to indicate errors,
+;; e.g. when we know that a function exists but the actual location is
+;; not known.
+(defclass xref-bogus-location (xref-location)
+  ((message :type string :initarg :message
+	    :reader xref-bogus-location-message)))
+
+(defun xref-make-bogus-location (message)
+  (make-instance 'xref-bogus-location :message message))
+
+(defmethod xref-location-buffer ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+(defmethod xref-location-position ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+\f
+;;; cross reference
+
+;; An xref is used to display and locate constructs like variables or
+;; functions.
+(defclass xref--xref ()
+  ((description :type string :initarg :description
+		:reader xref--xref-description)
+   (location :type xref-location :initarg :location
+	     :reader xref--xref-location)))
+
+(defun xref-make (description location)
+  (make-instance 'xref--xref :description description :location location))
+
+\f
+;;; Backend
+
+;; Setting the variable `xref-backend-function' to a function that
+;; returns a subclass of xref-backend-class can be used to provide
+;; language specific behaviour, primarily `xref-lookup-definitions'.
+
+;; Ugly name because defclass stores the class object in the symbol.
+(defclass xref-backend-class () ())
+
+;; For now, make the etags backend the default.
+(defvar xref-backend-function 'etags-xref-backend-function)
+
+(defun xref--backend ()
+  (funcall xref-backend-function))
+
+;;;; Backend interface functions
+
+(defgeneric xref-lookup-definitions (backend identifier))
+(defgeneric xref-lookup-references (backend identifier))
+
+;; An identifier is backend specific.  By default it's a string but it
+;; can be any type.  Well, nil means "no identifier at point" so that
+;; can't be used.
+(defgeneric xref-identifier-at-point (backend))
+(defgeneric xref-read-identifier-from-minibuffer (backend prompt default))
+(defgeneric xref-identifier-to-string (backend identifier))
+
+;; default implementation for identifiers
+(defmethod xref-identifier-at-point (_backend)
+  (let ((thing (thing-at-point 'symbol)))
+    (and thing (substring-no-properties thing))))
+
+(defmethod xref-read-identifier-from-minibuffer (_backend prompt init)
+  (read-from-minibuffer prompt init))
+
+(defmethod xref-identifier-to-string (_backend identifier)
+  (with-output-to-string (princ identifier)))
+
+\f
+;;; misc utilities
+(defun xref--alistify (list key test)
+  "Partition the elements of LIST into an alist.
+KEY extracts the key from an element and TEST is used to compare
+keys."
+  (let ((alist '()))
+    (dolist (e list)
+      (let* ((k (funcall key e))
+	     (probe (cl-assoc k alist :test test)))
+	(if probe
+	    (setcdr probe (cons e (cdr probe)))
+          (push (cons k (list e)) alist))))
+    ;; Put them back in order.
+    (cl-loop for (key . value) in (reverse alist)
+             collect (cons key (reverse value)))))
+
+(defun xref--insert-propertized (props &rest strings)
+  "Insert STRINGS with text properties PROPS."
+  (let ((start (point)))
+    (apply #'insert strings)
+    (add-text-properties start (point) props)))
+
+(defun xref--search-property (property &optional backward)
+    "Search the next text range where text property PROPERTY is non-nil.
+Return the value of PROPERTY.  If BACKWARD is non-nil, search
+backward."
+  (let ((next (if backward
+		  #'previous-single-char-property-change
+		#'next-single-char-property-change))
+        (start (point))
+        (value nil))
+    (while (progn
+             (goto-char (funcall next (point) property))
+             (not (or (setq value (get-text-property (point) property))
+                      (eobp)
+                      (bobp)))))
+    (cond (value)
+	  (t (goto-char start) nil))))
+
+\f
+;;; Marker stack  (M-. pushes, M-, pops)
+
+(defconst xref--marker-ring-length 16)
+
+(defvar xref--marker-ring (make-ring xref--marker-ring-length)
+  "Ring of markers to implement the marker stack.")
+
+(defun xref-push-marker-stack ()
+  "Add point to the marker stack."
+  (ring-insert xref--marker-ring (point-marker)))
+
+;;;###autoload
+(defun xref-pop-marker-stack ()
+  "Pop back to where \\[xref-find-definitions] was last invoked."
+  (interactive)
+  (let ((ring xref--marker-ring))
+    (when (ring-empty-p ring)
+      (error "Marker stack is empty"))
+    (let ((marker (ring-remove ring 0)))
+      (switch-to-buffer (or (marker-buffer marker)
+			    (error "The marked buffer has been deleted")))
+      (goto-char (marker-position marker))
+      (set-marker marker nil nil))))
+
+;; etags.el needs this
+(defun xref-clear-marker-stack ()
+  "Discard all markers from the marker stack."
+  (let ((ring xref--marker-ring))
+    (while (not (ring-empty-p ring))
+      (let ((marker (ring-remove ring)))
+	(set-marker marker nil nil)))))
+
+\f
+(defun xref--goto-location (location)
+  "Set buffer and point according to xref-location LOCATION."
+  (set-buffer (xref-location-buffer location))
+  (let ((pos (xref-location-position location)))
+    (cond ((and (<= (point-min) pos) (<= pos (point-max))))
+	  (widen-automatically (widen))
+	  (t (error "Location is outside accessible part of buffer")))
+    (goto-char pos)))
+
+(defun xref--pop-to-location (location &optional window)
+  "Goto xref-location LOCATION and display the buffer.
+WINDOW controls how the buffer is displayed:
+  nil      -- switch-to-buffer
+  'window  -- pop-to-buffer (other window)
+  'frame   -- pop-to-buffer (other frame)"
+  (xref--goto-location location)
+  (cl-ecase window
+    ((nil)  (switch-to-buffer (current-buffer)))
+    (window (pop-to-buffer (current-buffer) t))
+    (frame  (let ((pop-up-frames t)) (pop-to-buffer (current-buffer) t)))))
+
+\f
+;;; XREF buffer (part of the UI)
+
+;; The xref buffer is used to display a set of xrefs.
+
+(defun xref--display-position (pos other-window recenter-arg)
+  ;; show the location, but don't hijack focus.
+  (with-selected-window (display-buffer (current-buffer) other-window)
+    (goto-char pos)
+    (recenter recenter-arg)))
+
+(defgeneric xref--show-location (location))
+(defmethod xref--show-location ((l xref-bogus-location))
+  (with-slots (message) l
+    (message "%s" message)))
+
+(defmethod xref--show-location (location)
+  (xref--goto-location location)
+  (xref--display-position (point) t 1))
+
+(defun xref--next-line (backward)
+  (let ((loc (xref--search-property 'xref-location backward)))
+    (when loc
+      (xref--show-location loc))))
+
+(defun xref-next-line ()
+  "Move to the next xref and display its source in the other window."
+  (interactive)
+  (xref--next-line nil))
+
+(defun xref-prev-line ()
+  "Move to the previous xref and display its source in the other window."
+  (interactive)
+  (xref--next-line t))
+
+(defun xref--location-at-point ()
+  (or (get-text-property (point) 'xref-location)
+      (error "No reference at point.")))
+
+(defun xref-goto-xref ()
+  "Jump to the xref at point and close the xref buffer."
+  (interactive)
+  (xref--show-location (xref--location-at-point))
+  (quit-window))
+
+(define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
+  "Mode for displaying cross refenences."
+  (setq buffer-read-only t))
+
+(let ((map xref--xref-buffer-mode-map))
+  (define-key map (kbd "q") 'quit-window)
+  (define-key map [remap next-line] 'xref-next-line)
+  (define-key map [remap previous-line] 'xref-prev-line)
+  (define-key map (kbd "RET") 'xref-goto-xref)
+
+  ;; suggested by Johan Claesson "to further reduce finger movement":
+  (define-key map (kbd ".") 'xref-next-line)
+  (define-key map (kbd ",") 'xref-prev-line))
+
+(defun xref--buffer-name () "*xref*")
+
+(defun xref--insert-xrefs (xref-alist)
+  "Insert XREF-ALIST in the current-buffer.
+XREF-ALIST is of the form ((GROUP . (XREF ...)) ...).  Where
+GROUP is a string for decoration purposes and XREF is an
+`xref--xref' object."
+  (cl-loop for ((group . xrefs) . more1) on xref-alist do
+           (xref--insert-propertized '(face bold) group "\n")
+           (cl-loop for (xref . more2) on xrefs do
+		    (insert "  ")
+		    (with-slots (description location) xref
+		      (xref--insert-propertized
+		       (list 'xref-location location
+			     'face 'font-lock-keyword-face)
+		       description))
+		    (when (or more1 more2)
+		      (insert "\n")))))
+
+;; Return a string used to group a set of locations (this is typically
+;; the filename).
+(defgeneric xref-location-group (location))
+(defmethod xref-location-group ((_ xref-bogus-location)) "(No location)")
+(defmethod xref-location-group ((l xref-file-location))
+  (with-slots (file) l
+    file))
+(defmethod xref-location-group ((l xref-buffer-location))
+  (with-slots (buffer) l
+    (or (buffer-file-name buffer)
+	(format "(buffer %s)" (buffer-name buffer)))))
+
+(defun xref--analyze (xrefs)
+  "Find common filenames in XREFS.
+Return an alist of the form ((FILENAME . (XREF ...)) ...)."
+  (xref--alistify xrefs
+		  (lambda (x)
+		    (xref-location-group (xref--xref-location x)))
+		  #'equal))
+
+(defun xref--show-xref-buffer (xrefs)
+  (let ((xref-alist (xref--analyze xrefs)))
+    (with-current-buffer (get-buffer-create (xref--buffer-name))
+      (let ((inhibit-read-only t))
+	(erase-buffer)
+	(xref--insert-xrefs xref-alist)
+	(xref--xref-buffer-mode)
+	(pop-to-buffer (current-buffer))
+	(goto-char (point-min))
+	(current-buffer)))))
+
+\f
+;; This part of the UI seems fairly uncontroversial: it reads the
+;; identifier and deals with the single definition case.
+;;
+;; The controversial multiple definitions case is handed of to
+;; xref-show-xrefs-function.
+
+(defun xref--unique-location (xrefs)
+  "If it exists, return the single location in the list XREFS.
+If there are multiple or no locations in XREFS return nil."
+  (and xrefs
+       (let ((loc (xref--xref-location (car xrefs))))
+	 (and (cl-every (lambda (x)
+			  (xref-location= (xref--xref-location x) loc))
+			(cdr xrefs))
+	      loc))))
+
+(defvar xref-show-xrefs-function 'xref--show-xref-buffer
+  "Function to display a list of xrefs.")
+
+(defun xref--show-xrefs (id kind xrefs window)
+  (let ((1loc (xref--unique-location xrefs)))
+    (cond ((null xrefs)
+	   (error "No known %s for: %s"
+		  kind (xref-identifier-to-string (xref--backend) id)))
+	  (1loc
+	   (xref-push-marker-stack)
+	   (xref--pop-to-location 1loc window))
+	  (t
+	   (xref-push-marker-stack)
+	   (funcall xref-show-xrefs-function xrefs)))))
+
+(defun xref--read-identifier (prompt)
+  "Return the identifier at point or read it from the minibuffer."
+  (let* ((backend (xref--backend))
+	 (id (xref-identifier-at-point backend)))
+    (cond ((or current-prefix-arg (not id))
+	   (let ((init (if id (xref-identifier-to-string backend id))))
+	     (xref-read-identifier-from-minibuffer backend prompt init)))
+	  (t id))))
+
+\f
+;;; Commands
+
+(defun xref--find-definitions (id window)
+  (xref--show-xrefs id "definitions"
+		    (xref-lookup-definitions (xref--backend) id)
+		    window))
+
+;;;###autoload
+(defun xref-find-definitions (identifier)
+  "Find the definition of the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier nil))
+
+;;;###autoload
+(defun xref-find-definitions-other-window (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'window))
+
+;;;###autoload
+(defun xref-find-definitions-other-frame (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'frame))
+
+;;;###autoload
+(defun xref-find-references (identifier)
+  (interactive (list (xref--read-identifier "Find references of: ")))
+  (xref--show-xrefs identifier "references"
+		    (xref-lookup-references (xref--backend) identifier)
+		    nil))
+
+\f
+;;; Key bindings
+
+;;;###autoload (define-key esc-map "." 'xref-find-definitions)
+;;;###autoload (define-key esc-map "," 'xref-pop-marker-stack)
+;;;###autoload (define-key ctl-x-4-map "." 'xref-find-definitions-other-window)
+;;;###autoload (define-key ctl-x-5-map "." 'xref-find-definitions-other-frame)
+
+\f
+;;; ELisp backend
+
+;; This is defined here and not in elisp-mode.el so that we don't need
+;; to load xref.el just to create the *scratch* buffer.
+
+;; For now, this is just a wrapper around find-func.el in particular
+;; `find-definition-noselect'.
+
+(defclass xref--elisp-backend (xref-backend-class) ())
+
+(defvar xref--elisp-backend-var (make-instance 'xref--elisp-backend))
+
+;;;###autoload
+(defun xref-elisp-backend-function () xref--elisp-backend-var)
+
+(defun xref--elisp-find-definition (symbol type)
+  (let ((loc (condition-case err
+		 (let ((loc (save-excursion
+			      (find-definition-noselect symbol type))))
+		   (xref-make-buffer-location (car loc) (or (cdr loc) 1)))
+	       (error
+		(xref-make-bogus-location (error-message-string err)))))
+	(desc (format "(%s %s)" (or type 'defun) symbol)))
+    (xref-make desc loc)))
+
+;; FIXME: include other stuff likes faces, compiler-macros, methods...
+(defmethod xref-lookup-definitions ((_ xref--elisp-backend) id)
+  (let ((sym (intern-soft id)))
+    (if (null sym)
+	'()
+      (let ((fun (if (fboundp sym) (xref--elisp-find-definition sym nil)))
+	    (var (if (boundp sym) (xref--elisp-find-definition sym 'defvar))))
+	(remove nil (list fun var))))))
+
+\f
+(provide 'xref)
+
+;;; xref.el ends here
-- 
1.7.10.4

Re: Generalizing find-definition

[Stefan Monnier]

>> This said, I think the most urgent step is to get such a generalized
>> find-definition into Emacs.
> Here is a patch:

What happened to the other contenders?

> --- a/lisp/progmodes/elisp-mode.el
> +++ b/lisp/progmodes/elisp-mode.el
> @@ -231,6 +231,7 @@ Blank lines separate paragraphs.  Semicolons start comments.
>    (setq imenu-case-fold-search nil)
>    (setq-local eldoc-documentation-function
>                #'elisp-eldoc-documentation-function)
> +  (setq-local xref-backend-function 'xref-elisp-backend-function)
>    (add-hook 'completion-at-point-functions
>              #'elisp-completion-at-point nil 'local))

Hmm, so xref-elisp-backend-function is not in elisp-mode.el?
That's too bad.

> +(defvaralias 'find-tag-marker-ring 'xref--marker-ring)
> +(make-obsolete-variable 'find-tag-marker-ring nil "25.1")

You can use define-obsolete-variable-alias.

> +;; and `etags--xref-backend' classes for examples.
> +
> +(require 'cl-lib)

You need a ";;; Code:" up there.  Try C-u M-x checkdoc-current-buffer
(you don't have to heed all its recommendations, tho).

> +;; For now, make the etags backend the default.
> +(defvar xref-backend-function 'etags-xref-backend-function)

IIUC this is the main interface between xref and its backends, so it
very much needs a good docstring.

Also, I don't understand why it should be a function that returns
a "backend object" rather than being the backend object itself.


        Stefan

Re: Generalizing find-definition

[Helmut Eller]

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

On Mon, Dec 08 2014, Stefan Monnier wrote:

> What happened to the other contenders?

Nothing in particular.  The competition is still open.

>> +  (setq-local xref-backend-function 'xref-elisp-backend-function)
>>    (add-hook 'completion-at-point-functions
>>              #'elisp-completion-at-point nil 'local))
>
> Hmm, so xref-elisp-backend-function is not in elisp-mode.el?
> That's too bad.

The elisp backend code needs to define a subclass of xref-backend-class
and that can't be done without loading xref.el and eieio, I think.
elisp-mode is needed to create the *scratch* buffer and it seemed to me
that loading xref.el so early increases startup time for no good reason.

find-func.el might be a reasonable place for the elisp-xref backend
code.

>> +(defvaralias 'find-tag-marker-ring 'xref--marker-ring)
>> +(make-obsolete-variable 'find-tag-marker-ring nil "25.1")
>
> You can use define-obsolete-variable-alias.

OK.

>> +;; For now, make the etags backend the default.
>> +(defvar xref-backend-function 'etags-xref-backend-function)
>
> IIUC this is the main interface between xref and its backends, so it
> very much needs a good docstring.

I added a docstring here and in a few other places.

> Also, I don't understand why it should be a function that returns
> a "backend object" rather than being the backend object itself.

The indirection makes it easier to autoload xref.el.  If emacs-lisp-mode
wants to create a backend object it has to load xref.el first, which as
I said above seems undesirable.

Updated patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Generalized-infrastructure-for-find-definition.patch --]
[-- Type: text/x-diff, Size: 27172 bytes --]

From 031e27d9836aa6e0dc6223f3beaf3d0120e4007a Mon Sep 17 00:00:00 2001
From: Helmut Eller <eller.helmut@gmail.com>
Date: Mon, 8 Dec 2014 20:26:50 +0100
Subject: [PATCH] Generalized infrastructure for find-definition

* progmodes/xref.el: New file.

* progmodes/etags.el (find-tag-marker-ring, pop-tag-mark): Move to
xref but keep aliases for backward compatibility.
(tags-reset-tags-tables): Use xref marker stack instead of
find-tag-marker-ring.
(etags--xref-backend, etags--xref-backend-var)
(etags-xref-backend-function): New xref backend.
(esc-map, ctl-x-4-map, ctl-x-5-map): Move key bindings for M-.,
M-, C-x 4 M-., and C-x 5 M-. to xref.el

* progmodes/elisp-mode.el (emacs-lisp-mode): Initialize
xref-backend-function.
---
 lisp/ChangeLog               |   18 ++
 lisp/progmodes/elisp-mode.el |    1 +
 lisp/progmodes/etags.el      |   80 +++++--
 lisp/progmodes/xref.el       |  522 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 598 insertions(+), 23 deletions(-)
 create mode 100644 lisp/progmodes/xref.el

diff --git a/lisp/ChangeLog b/lisp/ChangeLog
index b3cb2fa..46baac5 100644
--- a/lisp/ChangeLog
+++ b/lisp/ChangeLog
@@ -1,3 +1,21 @@
+2014-12-07  Helmut Eller  <eller.helmut@gmail.com>
+
+	Generalized infrastructure for find-definition
+
+	* progmodes/xref.el: New file.
+
+	* progmodes/etags.el (find-tag-marker-ring, pop-tag-mark): Move to
+	xref but keep aliases for backward compatibility.
+	(tags-reset-tags-tables): Use xref marker stack instead of
+	find-tag-marker-ring.
+	(etags--xref-backend, etags--xref-backend-var)
+	(etags-xref-backend-function): New xref backend.
+	(esc-map, ctl-x-4-map, ctl-x-5-map): Move key bindings for M-.,
+	M-, C-x 4 M-., and C-x 5 M-. to xref.el
+
+	* progmodes/elisp-mode.el (emacs-lisp-mode): Initialize
+	xref-backend-function.
+
 2014-12-05  Juri Linkov  <juri@linkov.net>
 
 	* comint.el (comint-history-isearch-search)
diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ba70f90..b9caf69 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -231,6 +231,7 @@ Blank lines separate paragraphs.  Semicolons start comments.
   (setq imenu-case-fold-search nil)
   (setq-local eldoc-documentation-function
               #'elisp-eldoc-documentation-function)
+  (setq-local xref-backend-function #'xref-elisp-backend-function)
   (add-hook 'completion-at-point-functions
             #'elisp-completion-at-point nil 'local))
 
diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index b89b4cf..cd2e00b 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -28,6 +28,7 @@
 
 (require 'ring)
 (require 'button)
+(require 'xref)
 
 ;;;###autoload
 (defvar tags-file-name nil
@@ -182,8 +183,8 @@ Example value:
 		       (sexp :tag "Tags to search")))
   :version "21.1")
 
-(defvar find-tag-marker-ring (make-ring find-tag-marker-ring-length)
-  "Ring of markers which are locations from which \\[find-tag] was invoked.")
+(define-obsolete-variable-alias 'find-tag-marker-ring 'xref--marker-ring
+  "25.1")
 
 (defvar default-tags-table-function nil
   "If non-nil, a function to choose a default tags file for a buffer.
@@ -716,12 +717,10 @@ Returns t if it visits a tags table, or nil if there are no more in the list."
     (while (< i find-tag-marker-ring-length)
       (if (aref (cddr tags-location-ring) i)
 	  (set-marker (aref (cddr tags-location-ring) i) nil))
-      (if (aref (cddr find-tag-marker-ring) i)
-	  (set-marker (aref (cddr find-tag-marker-ring) i) nil))
       (setq i (1+ i))))
+  (xref-clear-marker-stack)
   (setq tags-file-name nil
 	tags-location-ring (make-ring find-tag-marker-ring-length)
-	find-tag-marker-ring (make-ring find-tag-marker-ring-length)
 	tags-table-list nil
 	tags-table-computed-list nil
 	tags-table-computed-list-for nil
@@ -898,7 +897,7 @@ See documentation of variable `tags-file-name'."
 	      ;; Run the user's hook.  Do we really want to do this for pop?
 	      (run-hooks 'local-find-tag-hook))))
       ;; Record whence we came.
-      (ring-insert find-tag-marker-ring (point-marker))
+      (xref-push-marker-stack)
       (if (and next-p last-tag)
 	  ;; Find the same table we last used.
 	  (visit-tags-table-buffer 'same)
@@ -954,7 +953,6 @@ See documentation of variable `tags-file-name'."
 	(switch-to-buffer buf)
       (error (pop-to-buffer buf)))
     (goto-char pos)))
-;;;###autoload (define-key esc-map "." 'find-tag)
 
 ;;;###autoload
 (defun find-tag-other-window (tagname &optional next-p regexp-p)
@@ -995,7 +993,6 @@ See documentation of variable `tags-file-name'."
 			;; the window's point from the buffer.
 			(set-window-point (selected-window) tagpoint))
 		      window-point)))
-;;;###autoload (define-key ctl-x-4-map "." 'find-tag-other-window)
 
 ;;;###autoload
 (defun find-tag-other-frame (tagname &optional next-p)
@@ -1020,7 +1017,6 @@ See documentation of variable `tags-file-name'."
   (interactive (find-tag-interactive "Find tag other frame: "))
   (let ((pop-up-frames t))
     (find-tag-other-window tagname next-p)))
-;;;###autoload (define-key ctl-x-5-map "." 'find-tag-other-frame)
 
 ;;;###autoload
 (defun find-tag-regexp (regexp &optional next-p other-window)
@@ -1049,20 +1045,8 @@ See documentation of variable `tags-file-name'."
 ;;;###autoload (define-key esc-map "*" 'pop-tag-mark)
 
 ;;;###autoload
-(defun pop-tag-mark ()
-  "Pop back to where \\[find-tag] was last invoked.
+(defalias 'pop-tag-mark 'xref-pop-marker-stack)
 
-This is distinct from invoking \\[find-tag] with a negative argument
-since that pops a stack of markers at which tags were found, not from
-where they were found."
-  (interactive)
-  (if (ring-empty-p find-tag-marker-ring)
-      (error "No previous locations for find-tag invocation"))
-  (let ((marker (ring-remove find-tag-marker-ring 0)))
-    (switch-to-buffer (or (marker-buffer marker)
-                          (error "The marked buffer has been deleted")))
-    (goto-char (marker-position marker))
-    (set-marker marker nil nil)))
 \f
 (defvar tag-lines-already-matched nil
   "Matches remembered between calls.") ; Doc string: calls to what?
@@ -1859,7 +1843,6 @@ nil, we exit; otherwise we scan the next file."
     (and messaged
 	 (null tags-loop-operate)
 	 (message "Scanning file %s...found" buffer-file-name))))
-;;;###autoload (define-key esc-map "," 'tags-loop-continue)
 
 ;;;###autoload
 (defun tags-search (regexp &optional file-list-form)
@@ -2077,6 +2060,57 @@ for \\[find-tag] (which see)."
       (completion-in-region (car comp-data) (cadr comp-data)
 			    (nth 2 comp-data)
 			    (plist-get (nthcdr 3 comp-data) :predicate)))))
+
+\f
+;;; Xref backed
+
+(defclass etags--xref-backend (xref-backend-class) ())
+
+(defvar etags--xref-backend-var (make-instance 'etags--xref-backend))
+
+;;;###autoload
+(defun etags-xref-backend-function () etags--xref-backend-var)
+
+(defmethod xref-lookup-definitions ((_ etags--xref-backend) id)
+  ;; This emulates the behaviour of `find-tag-in-order' but instead of
+  ;; returning one match at a time all matches are returned as list.
+  ;; NOTE: find-tag-tag-order is typically a buffer-local variable.
+  (let* ((xrefs '())
+	 (first-time t)
+	 (regexp? (consp id))
+	 (pattern (if regexp? (cadr id) id))
+	 (search-fun (if regexp? #'re-search-forward #'search-forward))
+	 (marks (make-hash-table :test 'equal)))
+    (save-excursion
+      (while (visit-tags-table-buffer (not first-time))
+	(setq first-time nil)
+	(dolist (order-fun (cond (regexp? find-tag-regexp-tag-order)
+				 (t find-tag-tag-order)))
+	  (goto-char (point-min))
+	  (while (funcall search-fun pattern nil t)
+	    (when (funcall order-fun pattern)
+	      (beginning-of-line)
+	      (cl-destructuring-bind (hint line &rest pos) (etags-snarf-tag)
+		(unless (eq hint t) ; hint==t if we are in a filename line
+		  (let* ((file (file-of-tag))
+			 (mark-key (cons file line)))
+		    (unless (gethash mark-key marks)
+		      (let ((loc (xref-make-file-location
+				  (expand-file-name file) line 0)))
+			(push (xref-make hint loc) xrefs)
+			(puthash mark-key t marks)))))))))))
+    (nreverse xrefs)))
+
+;; If the text in the minibuffer starts with " it's interpreted as a
+;; regexp.  This is an example for a non-trivial identifier type.
+(defmethod xref-read-identifier-from-minibuffer ((_ etags--xref-backend)
+						 prompt init)
+  (let ((string (read-from-minibuffer prompt init)))
+    (cond ((string-match "^\"" string)
+	   `(rx ,(read string)))
+	  (t
+	   string))))
+
 \f
 (provide 'etags)
 
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
new file mode 100644
index 0000000..6afc82d
--- /dev/null
+++ b/lisp/progmodes/xref.el
@@ -0,0 +1,522 @@
+;; xref.el --- Cross referencing commands              -*-lexical-binding:t-*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; This file provides a somewhat generic infrastructure for cross
+;; referencing commands, in particular "find-definition".  Some part of
+;; the functionality must be implemented in a language dependent way
+;; and that's done by defining a "backend".  The generic code finds
+;; the backend by calling the function stored in the variable
+;; `xref-backend-function'.  A language specific mode usually makes
+;; `xref-backend-function' buffer local before storing into it.
+;;
+;; A backend is an instance of the EIEIO class `xref-backend-class'.
+;; Various generic functions (in the EIEIO sense of the word) are
+;; defined on xref-backend-class.  A language specific mode usually
+;; creates a subclasses of xref-backend-class and provides specialized
+;; methods for the generic functions.  See the `xref--elisp-backend'
+;; and `etags--xref-backend' classes for examples.
+
+;;; Code:
+
+(require 'cl-lib)
+(require 'eieio)
+(require 'ring)
+(require 'find-func) ; for elisp backend
+
+\f
+;;; Locations
+
+(defclass xref-location () ()
+  :documentation "A location represents a position in a file or buffer.")
+
+;; If a backend decides to subclass xref-location it can provide
+;; methods for some of the following functions:
+(defgeneric xref-location-buffer (location)
+  "Return the buffer for LOCATION.")
+
+(defgeneric xref-location-position (location)
+  "Return the position in LOCATIONs buffer.")
+
+(defgeneric xref-location= (location1 location2)
+  "Return t if two locations are equal.")
+
+(defmethod xref-location= ((l1 xref-location) l2)
+  (equal l1 l2))
+
+;;;; Commonly needed location classes are defined here:
+
+;; FIXME: might be useful to have an optional "hint" i.e. a string to
+;; search for in case the line number is sightly out of date.
+(defclass xref-file-location (xref-location)
+  ((file :type string :initarg :file)
+   (line :type fixnum :initarg :line)
+   (column :type fixnum :initarg :column))
+  :documentation "A file location is a file/line/column triple.
+Line numbers start from 1 and columns from 0.")
+
+(defun xref-make-file-location (file line column)
+  "Create and return a new xref-file-location."
+  (make-instance 'xref-file-location :file file :line line :column column))
+
+(defmethod xref-location-buffer ((l xref-file-location))
+  (with-slots (file) l
+    (or (get-file-buffer file)
+	(let ((find-file-suppress-same-file-warnings t))
+	  (find-file-noselect file)))))
+
+(defmethod xref-location-position ((l xref-file-location))
+  (with-slots (line column) l
+    (with-current-buffer (xref-location-buffer l)
+      (save-restriction
+	(widen)
+	(save-excursion
+	  (goto-char (point-min))
+	  (beginning-of-line line)
+	  (move-to-column column)
+	  (point))))))
+
+(defclass xref-buffer-location (xref-location)
+  ((buffer :type buffer :initarg :buffer :reader xref-location-buffer)
+   (position :type fixnum :initarg :position :reader xref-location-position)))
+
+(defun xref-make-buffer-location (buffer position)
+  "Create and return a new xref-buffer-location."
+  (make-instance 'xref-buffer-location :buffer buffer :position position))
+
+(defclass xref-bogus-location (xref-location)
+  ((message :type string :initarg :message
+	    :reader xref-bogus-location-message))
+  :documentation "Bogus locations are sometimes useful to
+indicate errors, e.g. when we know that a function exists but the
+actual location is not known.")
+
+(defun xref-make-bogus-location (message)
+  "Create and return a new xref-bogus-location."
+  (make-instance 'xref-bogus-location :message message))
+
+(defmethod xref-location-buffer ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+(defmethod xref-location-position ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+\f
+;;; cross reference
+
+(defclass xref--xref ()
+  ((description :type string :initarg :description
+		:reader xref--xref-description)
+   (location :type xref-location :initarg :location
+	     :reader xref--xref-location))
+  :comment "An xref is used to display and locate constructs like
+variables or functions.")
+
+(defun xref-make (description location)
+  "Create and return an new xref.
+DESCRIPTION is a short string to describe the xref.
+LOCATION is an `xref-location'."
+  (make-instance 'xref--xref :description description :location location))
+
+\f
+;;; Backend
+
+;; Ugly name because defclass stores the class object in the symbol.
+(defclass xref-backend-class () ()
+  :documentation "Abstract superclass for backends.")
+
+;; For now, make the etags backend the default.
+(defvar xref-backend-function #'etags-xref-backend-function
+  "Function called to find the current xref-backend.
+The function is called with no arguments and should return
+a subclass of `xref-backend-class'.")
+
+(defun xref--backend ()
+  (funcall xref-backend-function))
+
+;;;; Backend interface functions
+
+(defgeneric xref-lookup-definitions (backend identifier)
+  "Find definitions of IDENTIFIER.
+The result is a list of `xref--xref' objects.
+If no definition can be found, return nil.")
+
+(defgeneric xref-lookup-references (backend identifier)
+  "Find references of IDENTIFIER.
+The result is a list of `xref--xref' objects.
+If no reference can be found, return nil.")
+
+;; An identifier is backend specific.  By default it's a string but it
+;; can be any type, expect nil.
+(defgeneric xref-identifier-at-point (backend)
+  "Search and return the identfier near point.
+If no identifier can be found, return nil.")
+
+(defgeneric xref-read-identifier-from-minibuffer (backend prompt init)
+  "Read an identifier from the minibuffer.
+PROMPT is a string used for prompting.
+INIT is either an identifier or nil.")
+
+(defgeneric xref-identifier-to-string (backend identifier)
+  "Return a string representing IDENTIFIER.")
+
+;; default implementation for identifiers
+(defmethod xref-identifier-at-point (_backend)
+  (let ((thing (thing-at-point 'symbol)))
+    (and thing (substring-no-properties thing))))
+
+(defmethod xref-read-identifier-from-minibuffer (backend prompt init)
+  (read-from-minibuffer prompt (xref-identifier-to-string backend init)))
+
+(defmethod xref-identifier-to-string (_backend identifier)
+  (with-output-to-string (princ identifier)))
+
+\f
+;;; misc utilities
+(defun xref--alistify (list key test)
+  "Partition the elements of LIST into an alist.
+KEY extracts the key from an element and TEST is used to compare
+keys."
+  (let ((alist '()))
+    (dolist (e list)
+      (let* ((k (funcall key e))
+	     (probe (cl-assoc k alist :test test)))
+	(if probe
+	    (setcdr probe (cons e (cdr probe)))
+          (push (cons k (list e)) alist))))
+    ;; Put them back in order.
+    (cl-loop for (key . value) in (reverse alist)
+             collect (cons key (reverse value)))))
+
+(defun xref--insert-propertized (props &rest strings)
+  "Insert STRINGS with text properties PROPS."
+  (let ((start (point)))
+    (apply #'insert strings)
+    (add-text-properties start (point) props)))
+
+(defun xref--search-property (property &optional backward)
+    "Search the next text range where text property PROPERTY is non-nil.
+Return the value of PROPERTY.  If BACKWARD is non-nil, search
+backward."
+  (let ((next (if backward
+		  #'previous-single-char-property-change
+		#'next-single-char-property-change))
+        (start (point))
+        (value nil))
+    (while (progn
+             (goto-char (funcall next (point) property))
+             (not (or (setq value (get-text-property (point) property))
+                      (eobp)
+                      (bobp)))))
+    (cond (value)
+	  (t (goto-char start) nil))))
+
+\f
+;;; Marker stack  (M-. pushes, M-, pops)
+
+(defconst xref--marker-ring-length 16)
+
+(defvar xref--marker-ring (make-ring xref--marker-ring-length)
+  "Ring of markers to implement the marker stack.")
+
+(defun xref-push-marker-stack ()
+  "Add point to the marker stack."
+  (ring-insert xref--marker-ring (point-marker)))
+
+;;;###autoload
+(defun xref-pop-marker-stack ()
+  "Pop back to where \\[xref-find-definitions] was last invoked."
+  (interactive)
+  (let ((ring xref--marker-ring))
+    (when (ring-empty-p ring)
+      (error "Marker stack is empty"))
+    (let ((marker (ring-remove ring 0)))
+      (switch-to-buffer (or (marker-buffer marker)
+			    (error "The marked buffer has been deleted")))
+      (goto-char (marker-position marker))
+      (set-marker marker nil nil))))
+
+;; etags.el needs this
+(defun xref-clear-marker-stack ()
+  "Discard all markers from the marker stack."
+  (let ((ring xref--marker-ring))
+    (while (not (ring-empty-p ring))
+      (let ((marker (ring-remove ring)))
+	(set-marker marker nil nil)))))
+
+\f
+(defun xref--goto-location (location)
+  "Set buffer and point according to xref-location LOCATION."
+  (set-buffer (xref-location-buffer location))
+  (let ((pos (xref-location-position location)))
+    (cond ((and (<= (point-min) pos) (<= pos (point-max))))
+	  (widen-automatically (widen))
+	  (t (error "Location is outside accessible part of buffer")))
+    (goto-char pos)))
+
+(defun xref--pop-to-location (location &optional window)
+  "Goto xref-location LOCATION and display the buffer.
+WINDOW controls how the buffer is displayed:
+  nil      -- switch-to-buffer
+  'window  -- pop-to-buffer (other window)
+  'frame   -- pop-to-buffer (other frame)"
+  (xref--goto-location location)
+  (cl-ecase window
+    ((nil)  (switch-to-buffer (current-buffer)))
+    (window (pop-to-buffer (current-buffer) t))
+    (frame  (let ((pop-up-frames t)) (pop-to-buffer (current-buffer) t)))))
+
+\f
+;;; XREF buffer (part of the UI)
+
+;; The xref buffer is used to display a set of xrefs.
+
+(defun xref--display-position (pos other-window recenter-arg)
+  ;; show the location, but don't hijack focus.
+  (with-selected-window (display-buffer (current-buffer) other-window)
+    (goto-char pos)
+    (recenter recenter-arg)))
+
+(defgeneric xref--show-location (location))
+(defmethod xref--show-location ((l xref-bogus-location))
+  (with-slots (message) l
+    (message "%s" message)))
+
+(defmethod xref--show-location (location)
+  (xref--goto-location location)
+  (xref--display-position (point) t 1))
+
+(defun xref--next-line (backward)
+  (let ((loc (xref--search-property 'xref-location backward)))
+    (when loc
+      (xref--show-location loc))))
+
+(defun xref-next-line ()
+  "Move to the next xref and display its source in the other window."
+  (interactive)
+  (xref--next-line nil))
+
+(defun xref-prev-line ()
+  "Move to the previous xref and display its source in the other window."
+  (interactive)
+  (xref--next-line t))
+
+(defun xref--location-at-point ()
+  (or (get-text-property (point) 'xref-location)
+      (error "No reference at point")))
+
+(defun xref-goto-xref ()
+  "Jump to the xref at point and close the xref buffer."
+  (interactive)
+  (xref--show-location (xref--location-at-point))
+  (quit-window))
+
+(define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
+  "Mode for displaying cross refenences."
+  (setq buffer-read-only t))
+
+(let ((map xref--xref-buffer-mode-map))
+  (define-key map (kbd "q") #'quit-window)
+  (define-key map [remap next-line] #'xref-next-line)
+  (define-key map [remap previous-line] #'xref-prev-line)
+  (define-key map (kbd "RET") #'xref-goto-xref)
+
+  ;; suggested by Johan Claesson "to further reduce finger movement":
+  (define-key map (kbd ".") #'xref-next-line)
+  (define-key map (kbd ",") #'xref-prev-line))
+
+(defun xref--buffer-name () "*xref*")
+
+(defun xref--insert-xrefs (xref-alist)
+  "Insert XREF-ALIST in the current-buffer.
+XREF-ALIST is of the form ((GROUP . (XREF ...)) ...).  Where
+GROUP is a string for decoration purposes and XREF is an
+`xref--xref' object."
+  (cl-loop for ((group . xrefs) . more1) on xref-alist do
+           (xref--insert-propertized '(face bold) group "\n")
+           (cl-loop for (xref . more2) on xrefs do
+		    (insert "  ")
+		    (with-slots (description location) xref
+		      (xref--insert-propertized
+		       (list 'xref-location location
+			     'face 'font-lock-keyword-face)
+		       description))
+		    (when (or more1 more2)
+		      (insert "\n")))))
+
+(defgeneric xref-location-group (location)
+  "Return a string used to group a set of locations.
+This is typically the filename.")
+
+(defmethod xref-location-group ((_ xref-bogus-location)) "(No location)")
+(defmethod xref-location-group ((l xref-file-location))
+  (with-slots (file) l
+    file))
+(defmethod xref-location-group ((l xref-buffer-location))
+  (with-slots (buffer) l
+    (or (buffer-file-name buffer)
+	(format "(buffer %s)" (buffer-name buffer)))))
+
+(defun xref--analyze (xrefs)
+  "Find common filenames in XREFS.
+Return an alist of the form ((FILENAME . (XREF ...)) ...)."
+  (xref--alistify xrefs
+		  (lambda (x)
+		    (xref-location-group (xref--xref-location x)))
+		  #'equal))
+
+(defun xref--show-xref-buffer (xrefs)
+  (let ((xref-alist (xref--analyze xrefs)))
+    (with-current-buffer (get-buffer-create (xref--buffer-name))
+      (let ((inhibit-read-only t))
+	(erase-buffer)
+	(xref--insert-xrefs xref-alist)
+	(xref--xref-buffer-mode)
+	(pop-to-buffer (current-buffer))
+	(goto-char (point-min))
+	(current-buffer)))))
+
+\f
+;; This part of the UI seems fairly uncontroversial: it reads the
+;; identifier and deals with the single definition case.
+;;
+;; The controversial multiple definitions case is handed of to
+;; xref-show-xrefs-function.
+
+(defun xref--unique-location (xrefs)
+  "If it exists, return the single location in the list XREFS.
+If there are multiple or no locations in XREFS return nil."
+  (and xrefs
+       (let ((loc (xref--xref-location (car xrefs))))
+	 (and (cl-every (lambda (x)
+			  (xref-location= (xref--xref-location x) loc))
+			(cdr xrefs))
+	      loc))))
+
+(defvar xref-show-xrefs-function 'xref--show-xref-buffer
+  "Function to display a list of xrefs.")
+
+(defun xref--show-xrefs (id kind xrefs window)
+  (let ((1loc (xref--unique-location xrefs)))
+    (cond ((null xrefs)
+	   (error "No known %s for: %s"
+		  kind (xref-identifier-to-string (xref--backend) id)))
+	  (1loc
+	   (xref-push-marker-stack)
+	   (xref--pop-to-location 1loc window))
+	  (t
+	   (xref-push-marker-stack)
+	   (funcall xref-show-xrefs-function xrefs)))))
+
+(defun xref--read-identifier (prompt)
+  "Return the identifier at point or read it from the minibuffer."
+  (let* ((backend (xref--backend))
+	 (id (xref-identifier-at-point backend)))
+    (cond ((or current-prefix-arg (not id))
+	   (xref-read-identifier-from-minibuffer backend prompt id))
+	  (t id))))
+
+\f
+;;; Commands
+
+(defun xref--find-definitions (id window)
+  (xref--show-xrefs id "definitions"
+		    (xref-lookup-definitions (xref--backend) id)
+		    window))
+
+;;;###autoload
+(defun xref-find-definitions (identifier)
+  "Find the definition of the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier nil))
+
+;;;###autoload
+(defun xref-find-definitions-other-window (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'window))
+
+;;;###autoload
+(defun xref-find-definitions-other-frame (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'frame))
+
+;;;###autoload
+(defun xref-find-references (identifier)
+  "Find references for the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find references of: ")))
+  (xref--show-xrefs identifier "references"
+		    (xref-lookup-references (xref--backend) identifier)
+		    nil))
+
+\f
+;;; Key bindings
+
+;;;###autoload
+(progn
+  (define-key esc-map "." #'xref-find-definitions)
+  (define-key esc-map "," #'xref-pop-marker-stack)
+  (define-key ctl-x-4-map "." #'xref-find-definitions-other-window)
+  (define-key ctl-x-5-map "." #'xref-find-definitions-other-frame))
+
+\f
+;;; ELisp backend
+
+;; This is defined here and not in elisp-mode.el so that we don't need
+;; to load xref.el just to create the *scratch* buffer.
+
+;; For now, this is just a wrapper around find-func.el in particular
+;; `find-definition-noselect'.
+
+(defclass xref--elisp-backend (xref-backend-class) ())
+
+(defvar xref--elisp-backend-var (make-instance 'xref--elisp-backend))
+
+;;;###autoload
+(defun xref-elisp-backend-function () xref--elisp-backend-var)
+
+(defun xref--elisp-find-definition (symbol type)
+  (let ((loc (condition-case err
+		 (let ((loc (save-excursion
+			      (find-definition-noselect symbol type))))
+		   (xref-make-buffer-location (car loc) (or (cdr loc) 1)))
+	       (error
+		(xref-make-bogus-location (error-message-string err)))))
+	(desc (format "(%s %s)" (or type 'defun) symbol)))
+    (xref-make desc loc)))
+
+;; FIXME: include other stuff likes faces, compiler-macros, methods...
+(defmethod xref-lookup-definitions ((_ xref--elisp-backend) id)
+  (let ((sym (intern-soft id)))
+    (if (null sym)
+	'()
+      (let ((fun (if (fboundp sym) (xref--elisp-find-definition sym nil)))
+	    (var (if (boundp sym) (xref--elisp-find-definition sym 'defvar))))
+	(remove nil (list fun var))))))
+
+\f
+(provide 'xref)
+
+;;; xref.el ends here
-- 
1.7.10.4

Re: Generalizing find-definition

[Stefan Monnier]

> > What happened to the other contenders?
> Nothing in particular.  The competition is still open.

Jorgen, what's your opinion?

> The elisp backend code needs to define a subclass of xref-backend-class
> and that can't be done without loading xref.el and eieio, I think.

Yes, I saw that.  I guess that's one advantage of the "plain function"
API compared to the EIEIO-based API.

> elisp-mode is needed to create the *scratch* buffer and it seemed to me
> that loading xref.el so early increases startup time for no good reason.

If it's needed for a normal "emacs -Q", then it should be preloaded.
And as it currently stands, I'd rather not preload EIEIO, so indeed the
elisp xref code can't be in elisp-mode.el.

> find-func.el might be a reasonable place for the elisp-xref
> backend code.

Indeed, that sounds like a good solution, thanks.

> The indirection makes it easier to autoload xref.el.

Hmm... I don't see why that'd make a difference, unless by "xref.el" you
mean "the file in which the elisp-xref-backend-function is defined".

> If emacs-lisp-mode wants to create a backend object it has to load
> xref.el first, which as I said above seems undesirable.

Indeed, I see that this makes a significant difference.

Another cosmetic issue I see with this EIEIO-based API is the need to
define a new class with a new instance which is really just a dummy
constant (no instance fields) and is only used to get the
method-dynamic-dispatch working (I guess in CLOS we could circumvent
this dummy class&instance by using an (eql <foo>) as the "class"
specializer).


        Stefan

Re: Generalizing find-definition

[Jorgen Schaefer]

On 12/08/14 22:38, Stefan Monnier wrote:
>>> What happened to the other contenders?
>> Nothing in particular.  The competition is still open.
>
> Jorgen, what's your opinion?

I was waiting for a decision by the project leader as to which of the 
proposed solutions is preferable before continuing to spend time on 
something that might not get used at all (question asked in 
<20141120212840.103e5758@forcix>).

I do not particularly care either way. I wanted a solution to a problem, 
and whether I write the solution myself or someone else does it is fine 
with me, as long as the problem is solved. :-)

Sadly, I will be mostly away from computers in the next weeks. If you 
decide for the function-based solution, I will not be able to work on it 
until after new year.

Regards,
Jorgen

Re: Generalizing find-definition

[Stefan Monnier]

> I do not particularly care either way. I wanted a solution to a problem, and
> whether I write the solution myself or someone else does it is fine with me,
> as long as the problem is solved. :-)

OK.  It looks like Helmut has the momentum, but in any case, thank you
very much for pushing this and making it happen.  It's been among those
things that have crossed my mind many times but I never got around
to it.


        Stefan

Re: Generalizing find-definition

[Stefan Monnier]

Alright, let's go ahead with this.  Can you send a "latest and greatest"
version of your code so I can install it into master?


        Stefan


>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> > What happened to the other contenders?
>> Nothing in particular.  The competition is still open.

> Jorgen, what's your opinion?

>> The elisp backend code needs to define a subclass of xref-backend-class
>> and that can't be done without loading xref.el and eieio, I think.

> Yes, I saw that.  I guess that's one advantage of the "plain function"
> API compared to the EIEIO-based API.

>> elisp-mode is needed to create the *scratch* buffer and it seemed to me
>> that loading xref.el so early increases startup time for no good reason.

> If it's needed for a normal "emacs -Q", then it should be preloaded.
> And as it currently stands, I'd rather not preload EIEIO, so indeed the
> elisp xref code can't be in elisp-mode.el.

>> find-func.el might be a reasonable place for the elisp-xref
>> backend code.

> Indeed, that sounds like a good solution, thanks.

>> The indirection makes it easier to autoload xref.el.

> Hmm... I don't see why that'd make a difference, unless by "xref.el" you
> mean "the file in which the elisp-xref-backend-function is defined".

>> If emacs-lisp-mode wants to create a backend object it has to load
>> xref.el first, which as I said above seems undesirable.

> Indeed, I see that this makes a significant difference.

> Another cosmetic issue I see with this EIEIO-based API is the need to
> define a new class with a new instance which is really just a dummy
> constant (no instance fields) and is only used to get the
> method-dynamic-dispatch working (I guess in CLOS we could circumvent
> this dummy class&instance by using an (eql <foo>) as the "class"
> specializer).


>         Stefan

Re: Generalizing find-definition

[Helmut Eller]

[-- Attachment #1: Type: text/plain, Size: 189 bytes --]

On Mon, Dec 08 2014, Stefan Monnier wrote:

> Alright, let's go ahead with this.  Can you send a "latest and greatest"
> version of your code so I can install it into master?

Here we go:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Generalized-infrastructure-for-find-definition.patch --]
[-- Type: text/x-diff, Size: 27397 bytes --]

From daa6a31e4903a5dd2d1450d4118fa0703ab88006 Mon Sep 17 00:00:00 2001
From: Helmut Eller <eller.helmut@gmail.com>
Date: Tue, 9 Dec 2014 09:37:34 +0100
Subject: [PATCH] Generalized infrastructure for find-definition

* progmodes/xref.el: New file.

* progmodes/etags.el (find-tag-marker-ring, pop-tag-mark): Move to
xref but keep aliases for backward compatibility.
(tags-reset-tags-tables): Use xref marker stack instead of
find-tag-marker-ring.
(etags--xref-backend, etags--xref-backend-var)
(etags-xref-backend-function): New xref backend.
(esc-map, ctl-x-4-map, ctl-x-5-map): Move key bindings for M-.,
M-,, C-x 4 M-., and C-x 5 M-. to xref.el

* emacs-lisp/find-func.el (find-function--xref-backend)
(find-function--xref-backend-var)
(find-function-xref-backend-function, find-function--find-xref):
New xref backend.

* progmodes/elisp-mode.el (emacs-lisp-mode): Initialize
xref-backend-function.
---
 lisp/emacs-lisp/find-func.el |   38 ++++
 lisp/progmodes/elisp-mode.el |    1 +
 lisp/progmodes/etags.el      |   88 ++++++--
 lisp/progmodes/xref.el       |  487 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 591 insertions(+), 23 deletions(-)
 create mode 100644 lisp/progmodes/xref.el

diff --git a/lisp/emacs-lisp/find-func.el b/lisp/emacs-lisp/find-func.el
index c372117..405135f 100644
--- a/lisp/emacs-lisp/find-func.el
+++ b/lisp/emacs-lisp/find-func.el
@@ -43,6 +43,8 @@
 
 ;;; Code:
 
+(require 'xref)
+
 ;;; User variables:
 
 (defgroup find-function nil
@@ -578,6 +580,42 @@ Set mark before moving, if the buffer already existed."
   (define-key ctl-x-4-map "V" 'find-variable-other-window)
   (define-key ctl-x-5-map "V" 'find-variable-other-frame))
 
+\f
+;;; Xref backend
+
+(defclass find-function--xref-backend (xref-backend-class) ())
+
+(defvar find-function--xref-backend-var
+  (make-instance 'find-function--xref-backend))
+
+;;;###autoload
+(defun find-function-xref-backend-function () find-function--xref-backend-var)
+
+(defun find-function--find-xref (symbol type)
+  (let ((loc (condition-case err
+		 (let ((loc (save-excursion
+			      (find-definition-noselect symbol type))))
+		   (xref-make-buffer-location (car loc) (or (cdr loc) 1)))
+	       (error
+		(xref-make-bogus-location (error-message-string err)))))
+	(desc (format "(%s %s)" (or type 'defun) symbol)))
+    (xref-make desc loc)))
+
+;; FIXME: include other stuff likes faces, compiler-macros, methods...
+(defmethod xref-lookup-definitions ((_ find-function--xref-backend) id)
+  (let ((sym (intern-soft id)))
+    (if (null sym)
+	'()
+      (let ((fun (if (fboundp sym) (find-function--find-xref sym nil)))
+	    (var (if (boundp sym) (find-function--find-xref sym 'defvar))))
+	(remove nil (list fun var))))))
+
+(defmethod xref-read-identifier-from-minibuffer
+  ((b find-function--xref-backend) prompt id)
+  (completing-read prompt obarray nil nil
+		   (if id (xref-identifier-to-string b id))))
+
+\f
 (provide 'find-func)
 
 ;;; find-func.el ends here
diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ba70f90..fa97890 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -231,6 +231,7 @@ Blank lines separate paragraphs.  Semicolons start comments.
   (setq imenu-case-fold-search nil)
   (setq-local eldoc-documentation-function
               #'elisp-eldoc-documentation-function)
+  (setq-local xref-backend-function #'find-function-xref-backend-function)
   (add-hook 'completion-at-point-functions
             #'elisp-completion-at-point nil 'local))
 
diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index b89b4cf..8696d8c 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -28,6 +28,7 @@
 
 (require 'ring)
 (require 'button)
+(require 'xref)
 
 ;;;###autoload
 (defvar tags-file-name nil
@@ -182,8 +183,8 @@ Example value:
 		       (sexp :tag "Tags to search")))
   :version "21.1")
 
-(defvar find-tag-marker-ring (make-ring find-tag-marker-ring-length)
-  "Ring of markers which are locations from which \\[find-tag] was invoked.")
+(define-obsolete-variable-alias 'find-tag-marker-ring 'xref--marker-ring
+  "25.1")
 
 (defvar default-tags-table-function nil
   "If non-nil, a function to choose a default tags file for a buffer.
@@ -716,12 +717,10 @@ Returns t if it visits a tags table, or nil if there are no more in the list."
     (while (< i find-tag-marker-ring-length)
       (if (aref (cddr tags-location-ring) i)
 	  (set-marker (aref (cddr tags-location-ring) i) nil))
-      (if (aref (cddr find-tag-marker-ring) i)
-	  (set-marker (aref (cddr find-tag-marker-ring) i) nil))
       (setq i (1+ i))))
+  (xref-clear-marker-stack)
   (setq tags-file-name nil
 	tags-location-ring (make-ring find-tag-marker-ring-length)
-	find-tag-marker-ring (make-ring find-tag-marker-ring-length)
 	tags-table-list nil
 	tags-table-computed-list nil
 	tags-table-computed-list-for nil
@@ -898,7 +897,7 @@ See documentation of variable `tags-file-name'."
 	      ;; Run the user's hook.  Do we really want to do this for pop?
 	      (run-hooks 'local-find-tag-hook))))
       ;; Record whence we came.
-      (ring-insert find-tag-marker-ring (point-marker))
+      (xref-push-marker-stack)
       (if (and next-p last-tag)
 	  ;; Find the same table we last used.
 	  (visit-tags-table-buffer 'same)
@@ -954,7 +953,6 @@ See documentation of variable `tags-file-name'."
 	(switch-to-buffer buf)
       (error (pop-to-buffer buf)))
     (goto-char pos)))
-;;;###autoload (define-key esc-map "." 'find-tag)
 
 ;;;###autoload
 (defun find-tag-other-window (tagname &optional next-p regexp-p)
@@ -995,7 +993,6 @@ See documentation of variable `tags-file-name'."
 			;; the window's point from the buffer.
 			(set-window-point (selected-window) tagpoint))
 		      window-point)))
-;;;###autoload (define-key ctl-x-4-map "." 'find-tag-other-window)
 
 ;;;###autoload
 (defun find-tag-other-frame (tagname &optional next-p)
@@ -1020,7 +1017,6 @@ See documentation of variable `tags-file-name'."
   (interactive (find-tag-interactive "Find tag other frame: "))
   (let ((pop-up-frames t))
     (find-tag-other-window tagname next-p)))
-;;;###autoload (define-key ctl-x-5-map "." 'find-tag-other-frame)
 
 ;;;###autoload
 (defun find-tag-regexp (regexp &optional next-p other-window)
@@ -1049,20 +1045,8 @@ See documentation of variable `tags-file-name'."
 ;;;###autoload (define-key esc-map "*" 'pop-tag-mark)
 
 ;;;###autoload
-(defun pop-tag-mark ()
-  "Pop back to where \\[find-tag] was last invoked.
+(defalias 'pop-tag-mark 'xref-pop-marker-stack)
 
-This is distinct from invoking \\[find-tag] with a negative argument
-since that pops a stack of markers at which tags were found, not from
-where they were found."
-  (interactive)
-  (if (ring-empty-p find-tag-marker-ring)
-      (error "No previous locations for find-tag invocation"))
-  (let ((marker (ring-remove find-tag-marker-ring 0)))
-    (switch-to-buffer (or (marker-buffer marker)
-                          (error "The marked buffer has been deleted")))
-    (goto-char (marker-position marker))
-    (set-marker marker nil nil)))
 \f
 (defvar tag-lines-already-matched nil
   "Matches remembered between calls.") ; Doc string: calls to what?
@@ -1859,7 +1843,6 @@ nil, we exit; otherwise we scan the next file."
     (and messaged
 	 (null tags-loop-operate)
 	 (message "Scanning file %s...found" buffer-file-name))))
-;;;###autoload (define-key esc-map "," 'tags-loop-continue)
 
 ;;;###autoload
 (defun tags-search (regexp &optional file-list-form)
@@ -2077,6 +2060,65 @@ for \\[find-tag] (which see)."
       (completion-in-region (car comp-data) (cadr comp-data)
 			    (nth 2 comp-data)
 			    (plist-get (nthcdr 3 comp-data) :predicate)))))
+
+\f
+;;; Xref backed
+
+(defclass etags--xref-backend (xref-backend-class) ())
+
+(defvar etags--xref-backend-var (make-instance 'etags--xref-backend))
+
+;;;###autoload
+(defun etags-xref-backend-function () etags--xref-backend-var)
+
+;; Stop searching if we find more than xref-limit matches, as the xref
+;; infrastracture is not designed to handle very long lists.
+;; Switching to some kind of lazy list might be better, but hopefully
+;; we hit the limit rarely.
+(defconst etags--xref-limit 1000)
+
+(defmethod xref-lookup-definitions ((_ etags--xref-backend) id)
+  ;; This emulates the behaviour of `find-tag-in-order' but instead of
+  ;; returning one match at a time all matches are returned as list.
+  ;; NOTE: find-tag-tag-order is typically a buffer-local variable.
+  (let* ((xrefs '())
+	 (first-time t)
+	 (regexp? (consp id))
+	 (pattern (if regexp? (cadr id) id))
+	 (search-fun (if regexp? #'re-search-forward #'search-forward))
+	 (marks (make-hash-table :test 'equal)))
+    (save-excursion
+      (while (visit-tags-table-buffer (not first-time))
+	(setq first-time nil)
+	(dolist (order-fun (cond (regexp? find-tag-regexp-tag-order)
+				 (t find-tag-tag-order)))
+	  (goto-char (point-min))
+	  (while (and (funcall search-fun pattern nil t)
+		      (< (hash-table-count marks) etags--xref-limit))
+	    (when (funcall order-fun pattern)
+	      (beginning-of-line)
+	      (cl-destructuring-bind (hint line &rest pos) (etags-snarf-tag)
+		(unless (eq hint t) ; hint==t if we are in a filename line
+		  (let* ((file (file-of-tag))
+			 (mark-key (cons file line)))
+		    (unless (gethash mark-key marks)
+		      (let ((loc (xref-make-file-location
+				  (expand-file-name file) line 0)))
+			(push (xref-make hint loc) xrefs)
+			(puthash mark-key t marks)))))))))))
+    (nreverse xrefs)))
+
+;; If the text in the minibuffer starts with " it's interpreted as a
+;; regexp.  This is an example for a non-trivial identifier type.
+(defmethod xref-read-identifier-from-minibuffer ((b etags--xref-backend)
+						 prompt id)
+  (let ((string (completing-read prompt (tags-lazy-completion-table) nil nil
+				 (if id (xref-identifier-to-string b id)))))
+    (cond ((string-match "^\"" string)
+	   `(rx ,(read string)))
+	  (t
+	   string))))
+
 \f
 (provide 'etags)
 
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
new file mode 100644
index 0000000..fd8d87b
--- /dev/null
+++ b/lisp/progmodes/xref.el
@@ -0,0 +1,487 @@
+;; xref.el --- Cross referencing commands              -*-lexical-binding:t-*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; This file provides a somewhat generic infrastructure for cross
+;; referencing commands, in particular "find-definition".  Some part of
+;; the functionality must be implemented in a language dependent way
+;; and that's done by defining a "backend".  The generic code finds
+;; the backend by calling the function stored in the variable
+;; `xref-backend-function'.  A language specific mode usually makes
+;; `xref-backend-function' buffer local before storing into it.
+;;
+;; A backend is an instance of the EIEIO class `xref-backend-class'.
+;; Various generic functions (in the EIEIO sense of the word) are
+;; defined on xref-backend-class.  A language specific mode usually
+;; creates a subclasses of xref-backend-class and provides specialized
+;; methods for the generic functions.  See the `etags--xref-backend'
+;; and `find-function--xref-backend' classes for examples.
+
+;;; Code:
+
+(require 'cl-lib)
+(require 'eieio)
+(require 'ring)
+
+\f
+;;; Locations
+
+(defclass xref-location () ()
+  :documentation "A location represents a position in a file or buffer.")
+
+;; If a backend decides to subclass xref-location it can provide
+;; methods for some of the following functions:
+(defgeneric xref-location-buffer (location)
+  "Return the buffer for LOCATION.")
+
+(defgeneric xref-location-position (location)
+  "Return the position in LOCATIONs buffer.")
+
+(defgeneric xref-location= (location1 location2)
+  "Return t if two locations are equal.")
+
+(defmethod xref-location= ((l1 xref-location) l2)
+  (equal l1 l2))
+
+;;;; Commonly needed location classes are defined here:
+
+;; FIXME: might be useful to have an optional "hint" i.e. a string to
+;; search for in case the line number is sightly out of date.
+(defclass xref-file-location (xref-location)
+  ((file :type string :initarg :file)
+   (line :type fixnum :initarg :line)
+   (column :type fixnum :initarg :column))
+  :documentation "A file location is a file/line/column triple.
+Line numbers start from 1 and columns from 0.")
+
+(defun xref-make-file-location (file line column)
+  "Create and return a new xref-file-location."
+  (make-instance 'xref-file-location :file file :line line :column column))
+
+(defmethod xref-location-buffer ((l xref-file-location))
+  (with-slots (file) l
+    (or (get-file-buffer file)
+	(let ((find-file-suppress-same-file-warnings t))
+	  (find-file-noselect file)))))
+
+(defmethod xref-location-position ((l xref-file-location))
+  (with-slots (line column) l
+    (with-current-buffer (xref-location-buffer l)
+      (save-restriction
+	(widen)
+	(save-excursion
+	  (goto-char (point-min))
+	  (beginning-of-line line)
+	  (move-to-column column)
+	  (point))))))
+
+(defclass xref-buffer-location (xref-location)
+  ((buffer :type buffer :initarg :buffer :reader xref-location-buffer)
+   (position :type fixnum :initarg :position :reader xref-location-position)))
+
+(defun xref-make-buffer-location (buffer position)
+  "Create and return a new xref-buffer-location."
+  (make-instance 'xref-buffer-location :buffer buffer :position position))
+
+(defclass xref-bogus-location (xref-location)
+  ((message :type string :initarg :message
+	    :reader xref-bogus-location-message))
+  :documentation "Bogus locations are sometimes useful to
+indicate errors, e.g. when we know that a function exists but the
+actual location is not known.")
+
+(defun xref-make-bogus-location (message)
+  "Create and return a new xref-bogus-location."
+  (make-instance 'xref-bogus-location :message message))
+
+(defmethod xref-location-buffer ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+(defmethod xref-location-position ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+\f
+;;; cross reference
+
+(defclass xref--xref ()
+  ((description :type string :initarg :description
+		:reader xref--xref-description)
+   (location :type xref-location :initarg :location
+	     :reader xref--xref-location))
+  :comment "An xref is used to display and locate constructs like
+variables or functions.")
+
+(defun xref-make (description location)
+  "Create and return an new xref.
+DESCRIPTION is a short string to describe the xref.
+LOCATION is an `xref-location'."
+  (make-instance 'xref--xref :description description :location location))
+
+\f
+;;; Backend
+
+;; Ugly name because defclass stores the class object in the symbol.
+(defclass xref-backend-class () ()
+  :documentation "Abstract superclass for backends.")
+
+;; For now, make the etags backend the default.
+(defvar xref-backend-function #'etags-xref-backend-function
+  "Function called to find the current xref-backend.
+The function is called with no arguments and should return
+a subclass of `xref-backend-class'.")
+
+(defun xref--backend ()
+  (funcall xref-backend-function))
+
+;;;; Backend interface functions
+
+(defgeneric xref-lookup-definitions (backend identifier)
+  "Find definitions of IDENTIFIER.
+The result is a list of `xref--xref' objects.
+If no definition can be found, return nil.")
+
+(defgeneric xref-lookup-references (backend identifier)
+  "Find references of IDENTIFIER.
+The result is a list of `xref--xref' objects.
+If no reference can be found, return nil.")
+
+;; An identifier is backend specific.  By default it's a string but it
+;; can be any type, expect nil.
+(defgeneric xref-identifier-at-point (backend)
+  "Search and return the identfier near point.
+If no identifier can be found, return nil.")
+
+(defgeneric xref-read-identifier-from-minibuffer (backend prompt init)
+  "Read an identifier from the minibuffer.
+PROMPT is a string used for prompting.
+INIT is either an identifier or nil.")
+
+(defgeneric xref-identifier-to-string (backend identifier)
+  "Return a string representing IDENTIFIER.")
+
+;; default implementation for identifiers
+(defmethod xref-identifier-at-point (_backend)
+  (let ((thing (thing-at-point 'symbol)))
+    (and thing (substring-no-properties thing))))
+
+(defmethod xref-read-identifier-from-minibuffer (backend prompt id)
+  (read-from-minibuffer prompt
+			(if id (xref-identifier-to-string backend id))))
+
+(defmethod xref-identifier-to-string (_backend identifier)
+  (with-output-to-string (princ identifier)))
+
+\f
+;;; misc utilities
+(defun xref--alistify (list key test)
+  "Partition the elements of LIST into an alist.
+KEY extracts the key from an element and TEST is used to compare
+keys."
+  (let ((alist '()))
+    (dolist (e list)
+      (let* ((k (funcall key e))
+	     (probe (cl-assoc k alist :test test)))
+	(if probe
+	    (setcdr probe (cons e (cdr probe)))
+          (push (cons k (list e)) alist))))
+    ;; Put them back in order.
+    (cl-loop for (key . value) in (reverse alist)
+             collect (cons key (reverse value)))))
+
+(defun xref--insert-propertized (props &rest strings)
+  "Insert STRINGS with text properties PROPS."
+  (let ((start (point)))
+    (apply #'insert strings)
+    (add-text-properties start (point) props)))
+
+(defun xref--search-property (property &optional backward)
+    "Search the next text range where text property PROPERTY is non-nil.
+Return the value of PROPERTY.  If BACKWARD is non-nil, search
+backward."
+  (let ((next (if backward
+		  #'previous-single-char-property-change
+		#'next-single-char-property-change))
+        (start (point))
+        (value nil))
+    (while (progn
+             (goto-char (funcall next (point) property))
+             (not (or (setq value (get-text-property (point) property))
+                      (eobp)
+                      (bobp)))))
+    (cond (value)
+	  (t (goto-char start) nil))))
+
+\f
+;;; Marker stack  (M-. pushes, M-, pops)
+
+(defconst xref--marker-ring-length 16)
+
+(defvar xref--marker-ring (make-ring xref--marker-ring-length)
+  "Ring of markers to implement the marker stack.")
+
+(defun xref-push-marker-stack ()
+  "Add point to the marker stack."
+  (ring-insert xref--marker-ring (point-marker)))
+
+;;;###autoload
+(defun xref-pop-marker-stack ()
+  "Pop back to where \\[xref-find-definitions] was last invoked."
+  (interactive)
+  (let ((ring xref--marker-ring))
+    (when (ring-empty-p ring)
+      (error "Marker stack is empty"))
+    (let ((marker (ring-remove ring 0)))
+      (switch-to-buffer (or (marker-buffer marker)
+			    (error "The marked buffer has been deleted")))
+      (goto-char (marker-position marker))
+      (set-marker marker nil nil))))
+
+;; etags.el needs this
+(defun xref-clear-marker-stack ()
+  "Discard all markers from the marker stack."
+  (let ((ring xref--marker-ring))
+    (while (not (ring-empty-p ring))
+      (let ((marker (ring-remove ring)))
+	(set-marker marker nil nil)))))
+
+\f
+(defun xref--goto-location (location)
+  "Set buffer and point according to xref-location LOCATION."
+  (set-buffer (xref-location-buffer location))
+  (let ((pos (xref-location-position location)))
+    (cond ((and (<= (point-min) pos) (<= pos (point-max))))
+	  (widen-automatically (widen))
+	  (t (error "Location is outside accessible part of buffer")))
+    (goto-char pos)))
+
+(defun xref--pop-to-location (location &optional window)
+  "Goto xref-location LOCATION and display the buffer.
+WINDOW controls how the buffer is displayed:
+  nil      -- switch-to-buffer
+  'window  -- pop-to-buffer (other window)
+  'frame   -- pop-to-buffer (other frame)"
+  (xref--goto-location location)
+  (cl-ecase window
+    ((nil)  (switch-to-buffer (current-buffer)))
+    (window (pop-to-buffer (current-buffer) t))
+    (frame  (let ((pop-up-frames t)) (pop-to-buffer (current-buffer) t)))))
+
+\f
+;;; XREF buffer (part of the UI)
+
+;; The xref buffer is used to display a set of xrefs.
+
+(defun xref--display-position (pos other-window recenter-arg)
+  ;; show the location, but don't hijack focus.
+  (with-selected-window (display-buffer (current-buffer) other-window)
+    (goto-char pos)
+    (recenter recenter-arg)))
+
+(defgeneric xref--show-location (location))
+(defmethod xref--show-location ((l xref-bogus-location))
+  (with-slots (message) l
+    (message "%s" message)))
+
+(defmethod xref--show-location (location)
+  (xref--goto-location location)
+  (xref--display-position (point) t 1))
+
+(defun xref--next-line (backward)
+  (let ((loc (xref--search-property 'xref-location backward)))
+    (when loc
+      (xref--show-location loc))))
+
+(defun xref-next-line ()
+  "Move to the next xref and display its source in the other window."
+  (interactive)
+  (xref--next-line nil))
+
+(defun xref-prev-line ()
+  "Move to the previous xref and display its source in the other window."
+  (interactive)
+  (xref--next-line t))
+
+(defun xref--location-at-point ()
+  (or (get-text-property (point) 'xref-location)
+      (error "No reference at point")))
+
+(defun xref-goto-xref ()
+  "Jump to the xref at point and close the xref buffer."
+  (interactive)
+  (xref--show-location (xref--location-at-point))
+  (quit-window))
+
+(define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
+  "Mode for displaying cross refenences."
+  (setq buffer-read-only t))
+
+(let ((map xref--xref-buffer-mode-map))
+  (define-key map (kbd "q") #'quit-window)
+  (define-key map [remap next-line] #'xref-next-line)
+  (define-key map [remap previous-line] #'xref-prev-line)
+  (define-key map (kbd "RET") #'xref-goto-xref)
+
+  ;; suggested by Johan Claesson "to further reduce finger movement":
+  (define-key map (kbd ".") #'xref-next-line)
+  (define-key map (kbd ",") #'xref-prev-line))
+
+(defun xref--buffer-name () "*xref*")
+
+(defun xref--insert-xrefs (xref-alist)
+  "Insert XREF-ALIST in the current-buffer.
+XREF-ALIST is of the form ((GROUP . (XREF ...)) ...).  Where
+GROUP is a string for decoration purposes and XREF is an
+`xref--xref' object."
+  (cl-loop for ((group . xrefs) . more1) on xref-alist do
+           (xref--insert-propertized '(face bold) group "\n")
+           (cl-loop for (xref . more2) on xrefs do
+		    (insert "  ")
+		    (with-slots (description location) xref
+		      (xref--insert-propertized
+		       (list 'xref-location location
+			     'face 'font-lock-keyword-face)
+		       description))
+		    (when (or more1 more2)
+		      (insert "\n")))))
+
+(defgeneric xref-location-group (location)
+  "Return a string used to group a set of locations.
+This is typically the filename.")
+
+(defmethod xref-location-group ((_ xref-bogus-location)) "(No location)")
+(defmethod xref-location-group ((l xref-file-location))
+  (with-slots (file) l
+    file))
+(defmethod xref-location-group ((l xref-buffer-location))
+  (with-slots (buffer) l
+    (or (buffer-file-name buffer)
+	(format "(buffer %s)" (buffer-name buffer)))))
+
+(defun xref--analyze (xrefs)
+  "Find common filenames in XREFS.
+Return an alist of the form ((FILENAME . (XREF ...)) ...)."
+  (xref--alistify xrefs
+		  (lambda (x)
+		    (xref-location-group (xref--xref-location x)))
+		  #'equal))
+
+(defun xref--show-xref-buffer (xrefs)
+  (let ((xref-alist (xref--analyze xrefs)))
+    (with-current-buffer (get-buffer-create (xref--buffer-name))
+      (let ((inhibit-read-only t))
+	(erase-buffer)
+	(xref--insert-xrefs xref-alist)
+	(xref--xref-buffer-mode)
+	(pop-to-buffer (current-buffer))
+	(goto-char (point-min))
+	(current-buffer)))))
+
+\f
+;; This part of the UI seems fairly uncontroversial: it reads the
+;; identifier and deals with the single definition case.
+;;
+;; The controversial multiple definitions case is handed of to
+;; xref-show-xrefs-function.
+
+(defun xref--unique-location (xrefs)
+  "If it exists, return the single location in the list XREFS.
+If there are multiple or no locations in XREFS return nil."
+  (and xrefs
+       (let ((loc (xref--xref-location (car xrefs))))
+	 (and (cl-every (lambda (x)
+			  (xref-location= (xref--xref-location x) loc))
+			(cdr xrefs))
+	      loc))))
+
+(defvar xref-show-xrefs-function 'xref--show-xref-buffer
+  "Function to display a list of xrefs.")
+
+(defun xref--show-xrefs (id kind xrefs window)
+  (let ((1loc (xref--unique-location xrefs)))
+    (cond ((null xrefs)
+	   (error "No known %s for: %s"
+		  kind (xref-identifier-to-string (xref--backend) id)))
+	  (1loc
+	   (xref-push-marker-stack)
+	   (xref--pop-to-location 1loc window))
+	  (t
+	   (xref-push-marker-stack)
+	   (funcall xref-show-xrefs-function xrefs)))))
+
+(defun xref--read-identifier (prompt)
+  "Return the identifier at point or read it from the minibuffer."
+  (let* ((backend (xref--backend))
+	 (id (xref-identifier-at-point backend)))
+    (cond ((or current-prefix-arg (not id))
+	   (xref-read-identifier-from-minibuffer backend prompt id))
+	  (t id))))
+
+\f
+;;; Commands
+
+(defun xref--find-definitions (id window)
+  (xref--show-xrefs id "definitions"
+		    (xref-lookup-definitions (xref--backend) id)
+		    window))
+
+;;;###autoload
+(defun xref-find-definitions (identifier)
+  "Find the definition of the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier nil))
+
+;;;###autoload
+(defun xref-find-definitions-other-window (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'window))
+
+;;;###autoload
+(defun xref-find-definitions-other-frame (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'frame))
+
+;;;###autoload
+(defun xref-find-references (identifier)
+  "Find references for the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find references of: ")))
+  (xref--show-xrefs identifier "references"
+		    (xref-lookup-references (xref--backend) identifier)
+		    nil))
+
+\f
+;;; Key bindings
+
+;;;###autoload
+(progn
+  (define-key esc-map "." #'xref-find-definitions)
+  (define-key esc-map "," #'xref-pop-marker-stack)
+  (define-key ctl-x-4-map "." #'xref-find-definitions-other-window)
+  (define-key ctl-x-5-map "." #'xref-find-definitions-other-frame))
+
+\f
+(provide 'xref)
+
+;;; xref.el ends here
-- 
1.7.10.4

Re: Generalizing find-definition

[Dmitry Gutov]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Dec 08 2014, Stefan Monnier wrote:
>
>> Alright, let's go ahead with this.  Can you send a "latest and greatest"
>> version of your code so I can install it into master?
>
> Here we go:

Would anyone mind if I try my hand at porting this code to the "plain
functions" approach?

The functionality is already nice, but the standard Emacs APIs probably
shouldn't force users to get familiar with EIEIO (even if it's not that
hard).

For the location value, using cl-defstruct seems to be
appropriate, but using plain functions and vars for everything else
should be simpler.

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Dec 09 2014, Dmitry Gutov wrote:

> Would anyone mind if I try my hand at porting this code to the "plain
> functions" approach?

I don't mind.  Rewrites welcome!

> For the location value, using cl-defstruct seems to be
> appropriate, but using plain functions and vars for everything else
> should be simpler.

Actually, the location stuff was the primary reason I started using
EIEIO.  In SLIME we have quite a zoo of location variants (not based on
EIEIO), e.g. file+byteoffset, file+charoffsets, search-strings, and
something that we call "source-paths".  A source-path describes a path
in the SEXP tree; sounds weird but works surprisingly well for Lisp
syntax.

Just want to say that locations should be very flexible.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 2981 bytes --]

Helmut Eller <eller.helmut@gmail.com> writes:

> I don't mind.  Rewrites welcome!

Ok, thanks. The modified proposal attached.

To be honest, now I can better see the appeal of using the classes here.
For instance, in the modified API a backend has to return `:none' in its
identifier-at-point implementation if not found, because otherwise the
default implementation will be called (and it's hard to implement the
overriding otherwise).

Although we could use a plist (or alist) instead of a
function-with-multiple-calling-conventions.

Suppose we end up using your implementation, do you think there could be
a reasonable case where a xref-backend-function creates a non-trivial
instance of its backend class, with instance vars, etc? Because
otherwise it'll remain an odd wart. Then the variable can just hold the
class symbol, and instantiation can be performed on-demain, or even each
time it's used: it's cheap anyway.

> Actually, the location stuff was the primary reason I started using
> EIEIO.  In SLIME we have quite a zoo of location variants (not based on
> EIEIO), e.g. file+byteoffset, file+charoffsets, search-strings, and
> something that we call "source-paths".  A source-path describes a path
> in the SEXP tree; sounds weird but works surprisingly well for Lisp
> syntax.
>
> Just want to say that locations should be very flexible.

Right, I can see how EIEIO can be useful in this case, so this is the
part I mostly left untouched, for now. However, it could be simplified:

* I'm pretty sure the default implementation of `xref-location=' will
  cover 99% of all cases.

* There's no real need for the above and `xref--unique-location'. We can
  just require the implementations to return lists without duplicates. I
  would think that most would do that even without being asked.

* What is `xref--show-location' for? Why do we want to allow certain
  types to override this behavior? I think `xref--show-xref-buffer' is
  also suspect at this stage of implementation.

`xref-location-group' is quite neat, though. But depending on whether we
really want to make this lighter-weight, the class hierarchy here can be
replaced with some ad-hoc polymorphism (so we could say that a location
can either be a (buffer position) (file line column), or, say, a closure
for the most general case).

-

I kept `etags--xref-read-identifier', but I don't particularly like it:
in other places, when we want to use a regexp, that's usually a
different command, or a prefix argument, etc. Inputting quotes to make
the contents work as a regexp is... odd.

-

The "identifier at point" and "read identifier" makes me wonder how it
would work with more complex languages, where we delegate a lot of logic
to an external server process. Will a two-request workflow become usual?
First ask the server what's at point, and then send it back with a
specific question. I guess some existing packages would be hard to adapt
to that.

-

We should probably port Jorgen's tests.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Modified xref proposal --]
[-- Type: text/x-diff, Size: 26546 bytes --]

diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ba70f90..d2b4b21 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -227,10 +227,12 @@ Blank lines separate paragraphs.  Semicolons start comments.
 
 \\{emacs-lisp-mode-map}"
   :group 'lisp
+  (defvar xref-backend)
   (lisp-mode-variables nil nil 'elisp)
   (setq imenu-case-fold-search nil)
   (setq-local eldoc-documentation-function
               #'elisp-eldoc-documentation-function)
+  (setq-local xref-backend #'elisp-xref-backend)
   (add-hook 'completion-at-point-functions
             #'elisp-completion-at-point nil 'local))
 
@@ -548,6 +550,38 @@ It can be quoted, or be inside a quoted form."
 (define-obsolete-function-alias
   'lisp-completion-at-point 'elisp-completion-at-point "25.1")
 
+;;; Xref backend
+
+(declare-function xref-make-buffer-location "xref" (buffer position))
+(declare-function xref-make-bogus-location "xref" (message))
+(declare-function xref-make "xref" (description location))
+
+(defun elisp-xref-backend (action &optional id &rest args)
+  (pcase action
+    (`definitions (elisp--xref-lookup-definitions id))
+    (`read-identifier (elisp--xref-read-identifier id (car args)))))
+
+(defun elisp--find-xref (symbol type)
+  (let ((loc (condition-case err
+                 (let ((loc (save-excursion
+                              (find-definition-noselect symbol type))))
+                   (xref-make-buffer-location (car loc) (or (cdr loc) 1)))
+               (error
+                (xref-make-bogus-location (error-message-string err)))))
+        (desc (format "(%s %s)" (or type 'defun) symbol)))
+    (xref-make desc loc)))
+
+;; FIXME: unify with `elisp--company-location'.
+(defun elisp--xref-lookup-definitions (id)
+  (let ((sym (intern-soft id)))
+    (when sym
+      (let ((fun (if (fboundp sym) (elisp--find-xref sym nil)))
+            (var (if (boundp sym) (elisp--find-xref sym 'defvar))))
+        (remove nil (list fun var))))))
+
+(defun elisp--xref-read-identifier (id prompt)
+  (completing-read prompt obarray nil nil id))
+
 ;;; Elisp Interaction mode
 
 (defvar lisp-interaction-mode-map
diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index b89b4cf..2a8a20d 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -28,6 +28,7 @@
 
 (require 'ring)
 (require 'button)
+(require 'xref)
 
 ;;;###autoload
 (defvar tags-file-name nil
@@ -182,8 +183,8 @@ Example value:
 		       (sexp :tag "Tags to search")))
   :version "21.1")
 
-(defvar find-tag-marker-ring (make-ring find-tag-marker-ring-length)
-  "Ring of markers which are locations from which \\[find-tag] was invoked.")
+(define-obsolete-variable-alias 'find-tag-marker-ring 'xref--marker-ring
+  "25.1")
 
 (defvar default-tags-table-function nil
   "If non-nil, a function to choose a default tags file for a buffer.
@@ -716,12 +717,10 @@ Returns t if it visits a tags table, or nil if there are no more in the list."
     (while (< i find-tag-marker-ring-length)
       (if (aref (cddr tags-location-ring) i)
 	  (set-marker (aref (cddr tags-location-ring) i) nil))
-      (if (aref (cddr find-tag-marker-ring) i)
-	  (set-marker (aref (cddr find-tag-marker-ring) i) nil))
       (setq i (1+ i))))
+  (xref-clear-marker-stack)
   (setq tags-file-name nil
 	tags-location-ring (make-ring find-tag-marker-ring-length)
-	find-tag-marker-ring (make-ring find-tag-marker-ring-length)
 	tags-table-list nil
 	tags-table-computed-list nil
 	tags-table-computed-list-for nil
@@ -898,7 +897,7 @@ See documentation of variable `tags-file-name'."
 	      ;; Run the user's hook.  Do we really want to do this for pop?
 	      (run-hooks 'local-find-tag-hook))))
       ;; Record whence we came.
-      (ring-insert find-tag-marker-ring (point-marker))
+      (xref-push-marker-stack)
       (if (and next-p last-tag)
 	  ;; Find the same table we last used.
 	  (visit-tags-table-buffer 'same)
@@ -954,7 +953,6 @@ See documentation of variable `tags-file-name'."
 	(switch-to-buffer buf)
       (error (pop-to-buffer buf)))
     (goto-char pos)))
-;;;###autoload (define-key esc-map "." 'find-tag)
 
 ;;;###autoload
 (defun find-tag-other-window (tagname &optional next-p regexp-p)
@@ -995,7 +993,6 @@ See documentation of variable `tags-file-name'."
 			;; the window's point from the buffer.
 			(set-window-point (selected-window) tagpoint))
 		      window-point)))
-;;;###autoload (define-key ctl-x-4-map "." 'find-tag-other-window)
 
 ;;;###autoload
 (defun find-tag-other-frame (tagname &optional next-p)
@@ -1020,7 +1017,6 @@ See documentation of variable `tags-file-name'."
   (interactive (find-tag-interactive "Find tag other frame: "))
   (let ((pop-up-frames t))
     (find-tag-other-window tagname next-p)))
-;;;###autoload (define-key ctl-x-5-map "." 'find-tag-other-frame)
 
 ;;;###autoload
 (defun find-tag-regexp (regexp &optional next-p other-window)
@@ -1049,20 +1045,8 @@ See documentation of variable `tags-file-name'."
 ;;;###autoload (define-key esc-map "*" 'pop-tag-mark)
 
 ;;;###autoload
-(defun pop-tag-mark ()
-  "Pop back to where \\[find-tag] was last invoked.
+(defalias 'pop-tag-mark 'xref-pop-marker-stack)
 
-This is distinct from invoking \\[find-tag] with a negative argument
-since that pops a stack of markers at which tags were found, not from
-where they were found."
-  (interactive)
-  (if (ring-empty-p find-tag-marker-ring)
-      (error "No previous locations for find-tag invocation"))
-  (let ((marker (ring-remove find-tag-marker-ring 0)))
-    (switch-to-buffer (or (marker-buffer marker)
-                          (error "The marked buffer has been deleted")))
-    (goto-char (marker-position marker))
-    (set-marker marker nil nil)))
 \f
 (defvar tag-lines-already-matched nil
   "Matches remembered between calls.") ; Doc string: calls to what?
@@ -1859,7 +1843,6 @@ nil, we exit; otherwise we scan the next file."
     (and messaged
 	 (null tags-loop-operate)
 	 (message "Scanning file %s...found" buffer-file-name))))
-;;;###autoload (define-key esc-map "," 'tags-loop-continue)
 
 ;;;###autoload
 (defun tags-search (regexp &optional file-list-form)
@@ -2077,6 +2060,63 @@ for \\[find-tag] (which see)."
       (completion-in-region (car comp-data) (cadr comp-data)
 			    (nth 2 comp-data)
 			    (plist-get (nthcdr 3 comp-data) :predicate)))))
+
+\f
+;;; Xref backend
+
+;; Stop searching if we find more than xref-limit matches, as the xref
+;; infrastracture is not designed to handle very long lists.
+;; Switching to some kind of lazy list might be better, but hopefully
+;; we hit the limit rarely.
+(defconst etags--xref-limit 1000)
+
+;;;###autoload
+(defun etags-xref-backend (action &optional id &rest args)
+  (pcase action
+    (`definitions (etags--xref-lookup-definitions id))
+    (`read-identifier (etags--xref-read-identifier id (car args)))))
+
+(defun etags--xref-lookup-definitions (id)
+  ;; This emulates the behaviour of `find-tag-in-order' but instead of
+  ;; returning one match at a time all matches are returned as list.
+  ;; NOTE: find-tag-tag-order is typically a buffer-local variable.
+  (let* ((xrefs '())
+         (first-time t)
+         (regexp? (consp id))
+         (pattern (if regexp? (cadr id) id))
+         (search-fun (if regexp? #'re-search-forward #'search-forward))
+         (marks (make-hash-table :test 'equal)))
+    (save-excursion
+      (while (visit-tags-table-buffer (not first-time))
+        (setq first-time nil)
+        (dolist (order-fun (cond (regexp? find-tag-regexp-tag-order)
+                                 (t find-tag-tag-order)))
+          (goto-char (point-min))
+          (while (and (funcall search-fun pattern nil t)
+                      (< (hash-table-count marks) etags--xref-limit))
+            (when (funcall order-fun pattern)
+              (beginning-of-line)
+              (cl-destructuring-bind (hint line &rest pos) (etags-snarf-tag)
+                (unless (eq hint t) ; hint==t if we are in a filename line
+                  (let* ((file (file-of-tag))
+                         (mark-key (cons file line)))
+                    (unless (gethash mark-key marks)
+                      (let ((loc (xref-make-file-location
+                                  (expand-file-name file) line 0)))
+                        (push (xref-make hint loc) xrefs)
+                        (puthash mark-key t marks)))))))))))
+    (nreverse xrefs)))
+
+;; If the text in the minibuffer starts with " it's interpreted as a
+;; regexp.  This is an example for a non-trivial identifier type.
+(defun etags--xref-read-identifier (id prompt)
+  (let ((string (completing-read prompt (tags-lazy-completion-table)
+                                 nil nil id)))
+    (cond ((string-match "\\`\"" string)
+           `(rx ,(read string)))
+          (t
+           string))))
+
 \f
 (provide 'etags)
 
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
new file mode 100644
index 0000000..f8c372e
--- /dev/null
+++ b/lisp/progmodes/xref.el
@@ -0,0 +1,486 @@
+;; xref.el --- Cross referencing commands              -*-lexical-binding:t-*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+
+
+;; This file provides a somewhat generic infrastructure for cross
+;; referencing commands, in particular "find-definition".  Some part
+;; of the functionality must be implemented in a language dependent
+;; way and that's done by defining a "backend".  It's a function,
+;; stored in the variable `xref-backend'.  A language-specific mode
+;; usually makes that variable buffer-local before storing into it.
+;;
+;; For `xref-backend' calling conventions, see its description.  In
+;; some cases, it has to operate with "xref" and "location" instances.
+;; One would usually call `make-xref' and `xref-make-file-location' or
+;; `xref-make-buffer-location' to create them.
+;;
+;; Another core notion is an identifier. Its representation is
+;; backend-specific.  By default it's a string but it can be any type.
+;; Two of the actions in `xref-backend' convert it to a string and back.
+;;
+;; See the functions `etags-xref-backend' and `elisp-xref-backend'
+;; for full examples.
+
+;;; Code:
+
+(require 'cl-lib)
+(require 'eieio)
+(require 'ring)
+
+\f
+;;; Locations
+
+(defclass xref-location () ()
+  :documentation "A location represents a position in a file or buffer.")
+
+;; If a backend decides to subclass xref-location it can provide
+;; methods for some of the following functions:
+(defgeneric xref-location-buffer (location)
+  "Return the buffer for LOCATION.")
+
+(defgeneric xref-location-position (location)
+  "Return the position in LOCATIONs buffer.")
+
+(defgeneric xref-location= (location1 location2)
+  "Return t if two locations are equal.")
+
+(defmethod xref-location= ((l1 xref-location) l2)
+  (equal l1 l2))
+
+;;;; Commonly needed location classes are defined here:
+
+;; FIXME: might be useful to have an optional "hint" i.e. a string to
+;; search for in case the line number is sightly out of date.
+(defclass xref-file-location (xref-location)
+  ((file :type string :initarg :file)
+   (line :type fixnum :initarg :line)
+   (column :type fixnum :initarg :column))
+  :documentation "A file location is a file/line/column triple.
+Line numbers start from 1 and columns from 0.")
+
+(defun xref-make-file-location (file line column)
+  "Create and return a new xref-file-location."
+  (make-instance 'xref-file-location :file file :line line :column column))
+
+(defmethod xref-location-buffer ((l xref-file-location))
+  (with-slots (file) l
+    (or (get-file-buffer file)
+        (let ((find-file-suppress-same-file-warnings t))
+          (find-file-noselect file)))))
+
+(defmethod xref-location-position ((l xref-file-location))
+  (with-slots (line column) l
+    (with-current-buffer (xref-location-buffer l)
+      (save-restriction
+        (widen)
+        (save-excursion
+          (goto-char (point-min))
+          (beginning-of-line line)
+          (move-to-column column)
+          (point))))))
+
+(defclass xref-buffer-location (xref-location)
+  ((buffer :type buffer :initarg :buffer :reader xref-location-buffer)
+   (position :type fixnum :initarg :position :reader xref-location-position)))
+
+(defun xref-make-buffer-location (buffer position)
+  "Create and return a new xref-buffer-location."
+  (make-instance 'xref-buffer-location :buffer buffer :position position))
+
+(defclass xref-bogus-location (xref-location)
+  ((message :type string :initarg :message
+            :reader xref-bogus-location-message))
+  :documentation "Bogus locations are sometimes useful to
+indicate errors, e.g. when we know that a function exists but the
+actual location is not known.")
+
+(defun xref-make-bogus-location (message)
+  "Create and return a new xref-bogus-location."
+  (make-instance 'xref-bogus-location :message message))
+
+(defmethod xref-location-buffer ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+(defmethod xref-location-position ((l xref-bogus-location))
+  (with-slots (message) l
+    (error "%s" message)))
+
+\f
+;;; cross reference
+
+(defclass xref--xref ()
+  ((description :type string :initarg :description
+                :reader xref--xref-description)
+   (location :type xref-location :initarg :location
+             :reader xref--xref-location))
+  :comment "An xref is used to display and locate constructs like
+variables or functions.")
+
+(defun xref-make (description location)
+  "Create and return a new xref.
+DESCRIPTION is a short string to describe the xref.
+LOCATION is an `xref-location'."
+  (make-instance 'xref--xref :description description :location location))
+
+\f
+;;; Backend
+
+;; For now, make the etags backend the default.
+(defvar xref-backend #'etags-xref-backend
+  "The current xref-backend.
+This function can be called with different sets of arguments, as
+described below.
+
+Required:
+
+ (definitions IDENTIFIER): Find definitions of IDENTIFIER.  The
+result must be a list of `xref--xref' objects.  If no definition
+can be found, return nil.
+
+ (references IDENTIFIER): Find references of IDENTIFIER.  The
+result must be a list of `xref--xref' objects.  If no reference
+can be found, return nil.
+
+Optional:
+
+ (read-identifier INIT PROMPT): Read an identifier from the
+minibuffer.  PROMPT is a string used for prompting.  INIT is
+either an identifier to use as the initial value or nil.
+
+ (identifier-at-point): Search and return the identfier near
+point.  If no identifier can be found, return `:none'.
+
+ (identifier-to-string IDENTIFIER): Return a string representing
+IDENTIFIER.")
+
+;;;; Backend interface functions
+
+(defun xref--call-backend (action &rest args)
+  (apply xref-backend action args))
+
+(defun xref--identifier-at-point ()
+  (let ((res (or (xref--call-backend 'identifier-at-point)
+                 (let ((thing (thing-at-point 'symbol)))
+                   (and thing (substring-no-properties thing))))))
+    (unless (eq res :none) res)))
+
+(defun xref--identifier-to-string (identifier)
+  (or (xref--call-backend 'identifier-to-string identifier)
+      (with-output-to-string (princ identifier))))
+
+\f
+;;; misc utilities
+(defun xref--alistify (list key test)
+  "Partition the elements of LIST into an alist.
+KEY extracts the key from an element and TEST is used to compare
+keys."
+  (let ((alist '()))
+    (dolist (e list)
+      (let* ((k (funcall key e))
+             (probe (cl-assoc k alist :test test)))
+        (if probe
+            (setcdr probe (cons e (cdr probe)))
+          (push (cons k (list e)) alist))))
+    ;; Put them back in order.
+    (cl-loop for (key . value) in (reverse alist)
+             collect (cons key (reverse value)))))
+
+(defun xref--insert-propertized (props &rest strings)
+  "Insert STRINGS with text properties PROPS."
+  (let ((start (point)))
+    (apply #'insert strings)
+    (add-text-properties start (point) props)))
+
+(defun xref--search-property (property &optional backward)
+    "Search the next text range where text property PROPERTY is non-nil.
+Return the value of PROPERTY.  If BACKWARD is non-nil, search
+backward."
+  (let ((next (if backward
+                  #'previous-single-char-property-change
+                #'next-single-char-property-change))
+        (start (point))
+        (value nil))
+    (while (progn
+             (goto-char (funcall next (point) property))
+             (not (or (setq value (get-text-property (point) property))
+                      (eobp)
+                      (bobp)))))
+    (cond (value)
+          (t (goto-char start) nil))))
+
+\f
+;;; Marker stack  (M-. pushes, M-, pops)
+
+(defconst xref--marker-ring-length 16)
+
+(defvar xref--marker-ring (make-ring xref--marker-ring-length)
+  "Ring of markers to implement the marker stack.")
+
+(defun xref-push-marker-stack ()
+  "Add point to the marker stack."
+  (ring-insert xref--marker-ring (point-marker)))
+
+;;;###autoload
+(defun xref-pop-marker-stack ()
+  "Pop back to where \\[xref-find-definitions] was last invoked."
+  (interactive)
+  (let ((ring xref--marker-ring))
+    (when (ring-empty-p ring)
+      (error "Marker stack is empty"))
+    (let ((marker (ring-remove ring 0)))
+      (switch-to-buffer (or (marker-buffer marker)
+                            (error "The marked buffer has been deleted")))
+      (goto-char (marker-position marker))
+      (set-marker marker nil nil))))
+
+;; etags.el needs this
+(defun xref-clear-marker-stack ()
+  "Discard all markers from the marker stack."
+  (let ((ring xref--marker-ring))
+    (while (not (ring-empty-p ring))
+      (let ((marker (ring-remove ring)))
+        (set-marker marker nil nil)))))
+
+\f
+(defun xref--goto-location (location)
+  "Set buffer and point according to xref-location LOCATION."
+  (set-buffer (xref-location-buffer location))
+  (let ((pos (xref-location-position location)))
+    (cond ((and (<= (point-min) pos) (<= pos (point-max))))
+          (widen-automatically (widen))
+          (t (error "Location is outside accessible part of buffer")))
+    (goto-char pos)))
+
+(defun xref--pop-to-location (location &optional window)
+  "Goto xref-location LOCATION and display the buffer.
+WINDOW controls how the buffer is displayed:
+  nil      -- switch-to-buffer
+  'window  -- pop-to-buffer (other window)
+  'frame   -- pop-to-buffer (other frame)"
+  (xref--goto-location location)
+  (cl-ecase window
+    ((nil)  (switch-to-buffer (current-buffer)))
+    (window (pop-to-buffer (current-buffer) t))
+    (frame  (let ((pop-up-frames t)) (pop-to-buffer (current-buffer) t)))))
+
+\f
+;;; XREF buffer (part of the UI)
+
+;; The xref buffer is used to display a set of xrefs.
+
+(defun xref--display-position (pos other-window recenter-arg)
+  ;; show the location, but don't hijack focus.
+  (with-selected-window (display-buffer (current-buffer) other-window)
+    (goto-char pos)
+    (recenter recenter-arg)))
+
+(defgeneric xref--show-location (location))
+(defmethod xref--show-location ((l xref-bogus-location))
+  (with-slots (message) l
+    (message "%s" message)))
+
+(defmethod xref--show-location (location)
+  (xref--goto-location location)
+  (xref--display-position (point) t 1))
+
+(defun xref--next-line (backward)
+  (let ((loc (xref--search-property 'xref-location backward)))
+    (when loc
+      (xref--show-location loc))))
+
+(defun xref-next-line ()
+  "Move to the next xref and display its source in the other window."
+  (interactive)
+  (xref--next-line nil))
+
+(defun xref-prev-line ()
+  "Move to the previous xref and display its source in the other window."
+  (interactive)
+  (xref--next-line t))
+
+(defun xref--location-at-point ()
+  (or (get-text-property (point) 'xref-location)
+      (error "No reference at point")))
+
+(defun xref-goto-xref ()
+  "Jump to the xref at point and close the xref buffer."
+  (interactive)
+  (xref--show-location (xref--location-at-point))
+  (quit-window))
+
+(define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
+  "Mode for displaying cross refenences."
+  (setq buffer-read-only t))
+
+(let ((map xref--xref-buffer-mode-map))
+  (define-key map (kbd "q") #'quit-window)
+  (define-key map [remap next-line] #'xref-next-line)
+  (define-key map [remap previous-line] #'xref-prev-line)
+  (define-key map (kbd "RET") #'xref-goto-xref)
+
+  ;; suggested by Johan Claesson "to further reduce finger movement":
+  (define-key map (kbd ".") #'xref-next-line)
+  (define-key map (kbd ",") #'xref-prev-line))
+
+(defun xref--buffer-name () "*xref*")
+
+(defun xref--insert-xrefs (xref-alist)
+  "Insert XREF-ALIST in the current-buffer.
+XREF-ALIST is of the form ((GROUP . (XREF ...)) ...).  Where
+GROUP is a string for decoration purposes and XREF is an
+`xref--xref' object."
+  (cl-loop for ((group . xrefs) . more1) on xref-alist do
+           (xref--insert-propertized '(face bold) group "\n")
+           (cl-loop for (xref . more2) on xrefs do
+                    (insert "  ")
+                    (with-slots (description location) xref
+                      (xref--insert-propertized
+                       (list 'xref-location location
+                             'face 'font-lock-keyword-face)
+                       description))
+                    (when (or more1 more2)
+                      (insert "\n")))))
+
+(defgeneric xref-location-group (location)
+  "Return a string used to group a set of locations.
+This is typically the filename.")
+
+(defmethod xref-location-group ((_ xref-bogus-location)) "(No location)")
+(defmethod xref-location-group ((l xref-file-location))
+  (with-slots (file) l
+    file))
+(defmethod xref-location-group ((l xref-buffer-location))
+  (with-slots (buffer) l
+    (or (buffer-file-name buffer)
+        (format "(buffer %s)" (buffer-name buffer)))))
+
+(defun xref--analyze (xrefs)
+  "Find common filenames in XREFS.
+Return an alist of the form ((FILENAME . (XREF ...)) ...)."
+  (xref--alistify xrefs
+                  (lambda (x)
+                    (xref-location-group (xref--xref-location x)))
+                  #'equal))
+
+(defun xref--show-xref-buffer (xrefs)
+  (let ((xref-alist (xref--analyze xrefs)))
+    (with-current-buffer (get-buffer-create (xref--buffer-name))
+      (let ((inhibit-read-only t))
+        (erase-buffer)
+        (xref--insert-xrefs xref-alist)
+        (xref--xref-buffer-mode)
+        (pop-to-buffer (current-buffer))
+        (goto-char (point-min))
+        (current-buffer)))))
+
+\f
+;; This part of the UI seems fairly uncontroversial: it reads the
+;; identifier and deals with the single definition case.
+;;
+;; The controversial multiple definitions case is handed of to
+;; xref-show-xrefs-function.
+
+(defun xref--unique-location (xrefs)
+  "If it exists, return the single location in the list XREFS.
+If there are multiple or no locations in XREFS return nil."
+  (and xrefs
+       (let ((loc (xref--xref-location (car xrefs))))
+         (and (cl-every (lambda (x)
+                          (xref-location= (xref--xref-location x) loc))
+                        (cdr xrefs))
+              loc))))
+
+(defvar xref-show-xrefs-function 'xref--show-xref-buffer
+  "Function to display a list of xrefs.")
+
+(defun xref--show-xrefs (id kind xrefs window)
+  (let ((1loc (xref--unique-location xrefs)))
+    (cond ((null xrefs)
+           (error "No known %s for: %s"
+                  kind (xref--identifier-to-string id)))
+          (1loc
+           (xref-push-marker-stack)
+           (xref--pop-to-location 1loc window))
+          (t
+           (xref-push-marker-stack)
+           (funcall xref-show-xrefs-function xrefs)))))
+
+(defun xref--read-identifier (prompt)
+  "Return the identifier at point or read it from the minibuffer."
+  (let ((id (xref--identifier-at-point)))
+    (cond ((or current-prefix-arg (not id))
+           (or (xref--call-backend 'read-identifier id prompt)
+               (read-from-minibuffer prompt
+                                     (if id (xref--identifier-to-string id)))))
+          (t id))))
+
+\f
+;;; Commands
+
+(defun xref--find-definitions (id window)
+  (xref--show-xrefs id "definitions"
+                    (xref--call-backend 'definitions id)
+                    window))
+
+;;;###autoload
+(defun xref-find-definitions (identifier)
+  "Find the definition of the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier nil))
+
+;;;###autoload
+(defun xref-find-definitions-other-window (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'window))
+
+;;;###autoload
+(defun xref-find-definitions-other-frame (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'frame))
+
+;;;###autoload
+(defun xref-find-references (identifier)
+  "Find references for the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find references of: ")))
+  (xref--show-xrefs identifier "references"
+                    (xref--call-backend 'references identifier)
+                    nil))
+
+\f
+;;; Key bindings
+
+;;;###autoload
+(progn
+  (define-key esc-map "." #'xref-find-definitions)
+  (define-key esc-map "," #'xref-pop-marker-stack)
+  (define-key ctl-x-4-map "." #'xref-find-definitions-other-window)
+  (define-key ctl-x-5-map "." #'xref-find-definitions-other-frame))
+
+\f
+(provide 'xref)
+
+;;; xref.el ends here

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 11 2014, Dmitry Gutov wrote:

> Ok, thanks. The modified proposal attached.

Looks pleasantly non-verbose.

> To be honest, now I can better see the appeal of using the classes here.
> For instance, in the modified API a backend has to return `:none' in its
> identifier-at-point implementation if not found, because otherwise the
> default implementation will be called (and it's hard to implement the
> overriding otherwise).

Maybe you could switch the roles around: return :not-implemented when a
backend has no implementation and make nil an ordinary return value.

> Suppose we end up using your implementation, do you think there could be
> a reasonable case where a xref-backend-function creates a non-trivial
> instance of its backend class, with instance vars, etc?

A cache or a socket connection to an external indexer could reasonably
be stored in a slot of a backend object.  But that is as far as my
imagination goes.

> otherwise it'll remain an odd wart. Then the variable can just hold the
> class symbol, and instantiation can be performed on-demain, or even each
> time it's used: it's cheap anyway.

Good idea!  Hadn't thought of that.

>> Just want to say that locations should be very flexible.
>
> Right, I can see how EIEIO can be useful in this case, so this is the
> part I mostly left untouched, for now.

Which kinda defeats the argument that API users shouldn't have to know
EIEIO.

> However, it could be simplified:
>
> * I'm pretty sure the default implementation of `xref-location=' will
>   cover 99% of all cases.
>
> * There's no real need for the above and `xref--unique-location'. We can
>   just require the implementations to return lists without duplicates. I
>   would think that most would do that even without being asked.

You're probably right.

> * What is `xref--show-location' for? Why do we want to allow certain
>   types to override this behavior?

xref-bogus-locations had to be handled somehow and using a GF is just as
good as a if/typecase statement.

>   I think `xref--show-xref-buffer' is
>   also suspect at this stage of implementation.

I'm not sure what you mean.

What I can see is that xref--show-xrefs should be have a cleaner arglist
and be part of the API, because there will be language specific commands
that would like to reuse the UI.

> `xref-location-group' is quite neat, though. But depending on whether we
> really want to make this lighter-weight, the class hierarchy here can be
> replaced with some ad-hoc polymorphism (so we could say that a location
> can either be a (buffer position) (file line column), or, say, a closure
> for the most general case).

Using closures as objects is always possible but rarely desirable.

> I kept `etags--xref-read-identifier', but I don't particularly like it:
> in other places, when we want to use a regexp, that's usually a
> different command, or a prefix argument, etc. Inputting quotes to make
> the contents work as a regexp is... odd.

ert-run-tests-interactively uses quotes to distinguish regexps from
symbols.  But I agree: a generalized apropos might be better than
forcing regexps into find-definition.

> The "identifier at point" and "read identifier" makes me wonder how it
> would work with more complex languages, where we delegate a lot of logic
> to an external server process. Will a two-request workflow become usual?
> First ask the server what's at point, and then send it back with a
> specific question. I guess some existing packages would be hard to adapt
> to that.

It seems to me that this situation is similar to a completion-at-point
command that asks an external process for possible completions.
Something that existing packages often do.  Despite that, nobody can
foresee all problems; instead we fix them as they come along.

Helmut

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 11 2014, Helmut Eller wrote:

>> otherwise it'll remain an odd wart. Then the variable can just hold the
>> class symbol, and instantiation can be performed on-demain, or even each
>> time it's used: it's cheap anyway.
>
> Good idea!  Hadn't thought of that.

Too bad that classes can't be autoloaded.

Helmut

Re: Generalizing find-definition

[Helmut Eller]

> Too bad that classes can't be autoloaded.

I added this little hack to autoload classes:

  (defvar xref-backend 'etags-xref-backend
    "The class symbol used to create backend instances.
  It should be a subclass of `xref-backend-class'.
  The front-end calls `make-instance' with this symbol as argument.")
  
  (defun xref--maybe-autoload-class (symbol)
    (unless (class-p symbol)
      (let ((f (symbol-function symbol)))
        (when (autoloadp f)
  	(autoload-do-load f symbol)))))
  
  (defun xref--backend ()
    (let ((symbol xref-backend))
      (xref--maybe-autoload-class symbol)
      (make-instance symbol)))

And an autoloadable class is then defined as:

  ;;;###autoload (autoload 'etags-xref-backend "etags")
  (defclass etags-xref-backend (xref-backend-class eieio-singleton) ())

This works because defclass creates a constructor function
with the same name as the classname.

Would this be acceptable, from a style point of view?

Helmut

Re: Generalizing find-definition

[David Engster]

Helmut Eller writes:
>> Too bad that classes can't be autoloaded.

They can. CEDET does that all the time.

>   ;;;###autoload (autoload 'etags-xref-backend "etags")
>   (defclass etags-xref-backend (xref-backend-class eieio-singleton) ())

Uhm, did you try to simply putting ";;;###autoload" before a class
definition? It should work just fine (through
`eieio-defclass-autoload').

However, there *is* a problem with autoloading classes, namely that
EIEIO should not be loaded at Emacs startup. This is why this feature is
currently restricted to the CEDET subsystem which is only loaded when
you activate Semantic or EDE.

-David

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 11 2014, David Engster wrote:

> Helmut Eller writes:
>>> Too bad that classes can't be autoloaded.
>
> They can. CEDET does that all the time.

Ah, that's great!

>>   ;;;###autoload (autoload 'etags-xref-backend "etags")
>>   (defclass etags-xref-backend (xref-backend-class eieio-singleton) ())
>
> Uhm, did you try to simply putting ";;;###autoload" before a class
> definition? It should work just fine (through
> `eieio-defclass-autoload').

Yes, I tried but got this error:

  ...
  Loading loaddefs.el (source)...
  Attempt to autoload eieio-defclass-autoload while preparing to dump
  make[1]: *** [emacs] Error 1

> However, there *is* a problem with autoloading classes, namely that
> EIEIO should not be loaded at Emacs startup. This is why this feature is
> currently restricted to the CEDET subsystem which is only loaded when
> you activate Semantic or EDE.

So what's the recommended way to do it for a file in lisp/progmodes/ ?

Helmut

Re: Generalizing find-definition

[David Engster]

Helmut Eller writes:
> On Thu, Dec 11 2014, David Engster wrote:
>
>> Helmut Eller writes:
>>>> Too bad that classes can't be autoloaded.
>>
>> They can. CEDET does that all the time.
>
> Ah, that's great!
>
>>>   ;;;###autoload (autoload 'etags-xref-backend "etags")
>>>   (defclass etags-xref-backend (xref-backend-class eieio-singleton) ())
>>
>> Uhm, did you try to simply putting ";;;###autoload" before a class
>> definition? It should work just fine (through
>> `eieio-defclass-autoload').
>
> Yes, I tried but got this error:
>
>   ...
>   Loading loaddefs.el (source)...
>   Attempt to autoload eieio-defclass-autoload while preparing to dump
>   make[1]: *** [emacs] Error 1

EIEIO is not available at startup. You have to explicitly 'require' it
before autoloading of classes can work.

>> However, there *is* a problem with autoloading classes, namely that
>> EIEIO should not be loaded at Emacs startup. This is why this feature is
>> currently restricted to the CEDET subsystem which is only loaded when
>> you activate Semantic or EDE.
>
> So what's the recommended way to do it for a file in lisp/progmodes/ ?

The only way is to separate everything that needs EIEIO into a separate
file with its own autoloads. This usually does not make sense for a
small package, though. So really, the best way is to have one central
autoload for your package (usually a major/minor mode).

-David

Re: Generalizing find-definition

[David Engster]

David Engster writes:
> The only way is to separate everything that needs EIEIO into a separate
> file with its own autoloads.

I just took a quick look at xref and understand a bit better what you're
trying to do. It's not about loading xref but the backends.

Your hack should work for your special case. If you look into what
eieio-defclass-autoload does, you'll see that the general case is more
complicated, because you might need to set up superclasses as well.

If your hack is acceptable is up to Stefan, of course...

-David

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 11 2014, David Engster wrote:

> David Engster writes:
>> The only way is to separate everything that needs EIEIO into a separate
>> file with its own autoloads.
>
> I just took a quick look at xref and understand a bit better what you're
> trying to do. It's not about loading xref but the backends.
>
> Your hack should work for your special case. If you look into what
> eieio-defclass-autoload does, you'll see that the general case is more
> complicated, because you might need to set up superclasses as well.
>
> If your hack is acceptable is up to Stefan, of course...

Thanks for the answers.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/11/2014 10:09 AM, Helmut Eller wrote:

> Looks pleasantly non-verbose.

Thanks, that was one of the goals.

> Maybe you could switch the roles around: return :not-implemented when a
> backend has no implementation and make nil an ordinary return value.

That's an option. But if we require that for every action, it'll 
increase verbosity. We only care about the distinction between nil and 
"nothing" in `identifier-at-point', so far, so that would be suboptimal too.

If we have a separate identifier-at-point-function, like Stefan wrote in 
the other message, this problem largely disappears. Kind obvious, in 
hindsight.

>> otherwise it'll remain an odd wart. Then the variable can just hold the
>> class symbol, and instantiation can be performed on-demain, or even each
>> time it's used: it's cheap anyway.
>
> Good idea!  Hadn't thought of that.
> ...
 > Too bad that classes can't be autoloaded.

Right. I don't have a better suggestion here, sorry.

>> Right, I can see how EIEIO can be useful in this case, so this is the
>> part I mostly left untouched, for now.
>
> Which kinda defeats the argument that API users shouldn't have to know
> EIEIO.

Not necessarily. The user-facing part of the API is plain functions now, 
and they can also use the xref-make-*-location function without really 
caring what they return. It's only when a user wants to add new location 
types they will really need to use the EIEIO API.

But of course, it would have to be loaded at runtime.

>> * What is `xref--show-location' for? Why do we want to allow certain
>>    types to override this behavior?
>
> xref-bogus-locations had to be handled somehow and using a GF is just as
> good as a if/typecase statement.

I think that gives it too much responsibility. What do we want from a 
location? Chiefly, a buffer-position pair.

* We don't need separate location-buffer and location-position 
accessors. Let's just have a method that returns a cons.

* Bogus locations will return nil and show a message. Or maybe signal an 
error, if you like. We can define a special kind of error, 
`xref--show-location' (a plain function) will catch it and display the 
error message, if it's so inclined. Another option is a special return 
value, like (:error . "Boo hoo").

Together with xref-location-group, this leaves just two methods. And 
without that one, it could be implemented as something like a future, I 
guess.

>>    I think `xref--show-xref-buffer' is
>>    also suspect at this stage of implementation.
>
> I'm not sure what you mean.

I mean it's fine as a possibility, but we don't have any other 
implementations, and thus don't really know what API they'd require.

If it were up to me, I'd remove it, to maybe bring back later, after the 
presently discussed feature is settled, and Stephen has experimented 
with a compilation-mode implementation.

> What I can see is that xref--show-xrefs should be have a cleaner arglist
> and be part of the API, because there will be language specific commands
> that would like to reuse the UI.

Ok, maybe. It should provide some definite benefit, though, over just 
doing (funcall xref-show-xrefs-function xrefs).

> Using closures as objects is always possible but rarely desirable.

Instead of a (usually opaque) closure, it could be a struct with a 
function reference (usually symbol) and an arguments list, which would 
be passed to it together with some changing argument. That would be 
quite transparent.

But anyway, I don't mind using EIEIO either.

> ert-run-tests-interactively uses quotes to distinguish regexps from
> symbols.  But I agree: a generalized apropos might be better than
> forcing regexps into find-definition.

I guess that would just mean another action for the 
xref-backend-function, in terms of the API?

> It seems to me that this situation is similar to a completion-at-point
> command that asks an external process for possible completions.
> Something that existing packages often do.  Despite that, nobody can
> foresee all problems; instead we fix them as they come along.

Similar, yes. With completion-at-point, the backend sees the current 
buffer and point, but also the completion table can determine where in 
the buffer the current "prefix" begins, which is usually important for 
external tools.

If the identifier-at-point is just a string (or some value that can be 
derived from a string), that would be limiting.

Re: Generalizing find-definition

[Stefan Monnier]

> * We don't need separate location-buffer and location-position
> accessors. Let's just have a method that returns a cons.

Why not a marker?


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/12/2014 01:55 AM, Stefan Monnier wrote:
>> * We don't need separate location-buffer and location-position
>> accessors. Let's just have a method that returns a cons.
>
> Why not a marker?

Sure, sounds good.

Re: Generalizing find-definition

[Stefan Monnier]

> To be honest, now I can better see the appeal of using the classes here.
> For instance, in the modified API a backend has to return `:none' in its
> identifier-at-point implementation if not found, because otherwise the
> default implementation will be called (and it's hard to implement the
> overriding otherwise).

You don't need :none.
You can do:

   (add-function :around (local xref-backend) #'elisp-xref-backend)

and then in elisp-xref-backend you do:

   (defun elisp-xref-backend (orig action ...)
     (pcase action
       ...
       (_ (apply orig args))))

> +;; For now, make the etags backend the default.
> +(defvar xref-backend #'etags-xref-backend

This should be called `xref-backend-function', to follow usual practice.

> +  "The current xref-backend.
> +This function can be called with different sets of arguments, as
> +described below.
> +
> +Required:
> +
> + (definitions IDENTIFIER): Find definitions of IDENTIFIER.  The
> +result must be a list of `xref--xref' objects.  If no definition
> +can be found, return nil.

Rename xref--xref to xref-xref, since it's clearly part of the API.

> + (references IDENTIFIER): Find references of IDENTIFIER.  The
> +result must be a list of `xref--xref' objects.  If no reference
> +can be found, return nil.
> +
> +Optional:
> +
> + (read-identifier INIT PROMPT): Read an identifier from the
> +minibuffer.  PROMPT is a string used for prompting.  INIT is
> +either an identifier to use as the initial value or nil.
> +
> + (identifier-at-point): Search and return the identfier near
> +point.  If no identifier can be found, return `:none'.
> +
> + (identifier-to-string IDENTIFIER): Return a string representing
> +IDENTIFIER.")

A few comments:
- As discussed earlier, `read-identifier' is probably a bad idea.
  Replace it with a `completion-table' method, so the completing-read is
  not in the backend but in xref.el.
- identifier-at-point should document more clearly that the return value
  may be something else than a string.  AFAIK the only interesting
  non-string non-nil case is to return a special value (e.g. a marker
  or just `t' to stand to (point)) which stands for "let the
  find-definitions code figure out the identifier at that place".
- If we restrict identifier to "a string or nil or t", then we can get
  rid of identifier-to-string.
- The first two methods use sufficiently similar (actual, identical)
  calling conventions, so it's perfectly fine to make them share
  a single backend function, but adding the other methods makes it
  scream "I want to use OO", just like completion-tables do.
  IOW, I think if we don't want to use EIEIO for the `xref-backend' objects,
  then we need to split `xref-backend' into several variables:
  - xref-find-function (can be used both for the `definitions' and for
    the `references').
  - xref-identifier-at-point-function.
  - xref-identifier-completion-table.


-- Stefan

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 11 2014, Stefan Monnier wrote:

> - If we restrict identifier to "a string or nil or t", then we can get
>   rid of identifier-to-string.

Doesn't look like a great idea to me.  identifier-to-string is used to
create error messages like: "No known definitions for: fooo".  How would
the message look like for the t case?

Helmut

Re: Generalizing find-definition

[Stefan Monnier]

>> - If we restrict identifier to "a string or nil or t", then we can get
>> rid of identifier-to-string.
> Doesn't look like a great idea to me.  identifier-to-string is used to
> create error messages like: "No known definitions for: fooo".  How would
> the message look like for the t case?

The message could look like "No known definition for ident at point".

The use case for `t' is when the backend doesn't even know how to name
the identifier at point.  It just passes the whole file along with the
position of point to some external program which returns the possible
definitions.  So there really might really be no good string
representation, regardless of our API (maybe the external program could
give us some kind of textual representation name for it, but then again
maybe it doesn't support this functionality and maybe this description
would be too long/complex).


        Stefan

Re: Generalizing find-definition

[Helmut Eller]

On Thu, Dec 11 2014, Stefan Monnier wrote:

>>> - If we restrict identifier to "a string or nil or t", then we can get
>>> rid of identifier-to-string.
>> Doesn't look like a great idea to me.  identifier-to-string is used to
>> create error messages like: "No known definitions for: fooo".  How would
>> the message look like for the t case?
>
> The message could look like "No known definition for ident at point".
>
> The use case for `t' is when the backend doesn't even know how to name
> the identifier at point.  It just passes the whole file along with the
> position of point to some external program which returns the possible
> definitions.  So there really might really be no good string
> representation, regardless of our API (maybe the external program could
> give us some kind of textual representation name for it, but then again
> maybe it doesn't support this functionality and maybe this description
> would be too long/complex).

A identifier representation like 

 (list (thing-at-point 'line) (current-buffer) (point))

doesn't cost anything and would go a long way to give a better error
message.  If thing-at-point returns nil or an empty line then
identifier-at-point should just return nil.

Helmut

Re: Generalizing find-definition

[Stefan Monnier]

> A identifier representation like 
>  (list (thing-at-point 'line) (current-buffer) (point))

If identifier-at-point returns t you can use the above as
an approximation if you need to generate a message for it, but it won't
be much more helpful to the user.


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

Helmut Eller <eller.helmut@gmail.com> writes:

> A identifier representation like 
>
>  (list (thing-at-point 'line) (current-buffer) (point))
>
> doesn't cost anything and would go a long way to give a better error
> message.  If thing-at-point returns nil or an empty line then
> identifier-at-point should just return nil.

Bearing in mind the example I mentioned in the previous email, maybe we
can add an optional argument to the xref-identifier-at-point-function.

When it's t, the function would return an approximate string (basically
symbol at point, but maybe with minor syntax adjustments), and that
value would be used both in the error message (when we can't find
anything), and as initial input for identifier completion, from
xref-identifier-completion-table.

Re: Generalizing find-definition

[Stefan Monnier]

> When it's t, the function would return an approximate string (basically
> symbol at point, but maybe with minor syntax adjustments), and that
> value would be used both in the error message (when we can't find
> anything), and as initial input for identifier completion, from
> xref-identifier-completion-table.

I don't care about which values are really allowed as identifiers.
I just the docstring to say clearly that non-strings can be used.
And I'd like to get rid of the "identifier->string" method because
I think it's over-engineering (it's only used to give
very-slightly-prettier error messages).


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> When it's t, the function would return an approximate string (basically
>> symbol at point, but maybe with minor syntax adjustments), and that
>> value would be used both in the error message (when we can't find
>> anything), and as initial input for identifier completion, from
>> xref-identifier-completion-table.
>
> I don't care about which values are really allowed as identifiers.
> I just the docstring to say clearly that non-strings can be used.
> And I'd like to get rid of the "identifier->string" method because
> I think it's over-engineering (it's only used to give
> very-slightly-prettier error messages).

The message you're responding to assumes familiarity with the patch,
which, I think, already addresses both your points (and identifiers can
only be strings, nil or t, like you suggested before).

Re: Generalizing find-definition

[Helmut Eller]

On Mon, Dec 15 2014, Dmitry Gutov wrote:

> Helmut Eller <eller.helmut@gmail.com> writes:
>
>> A identifier representation like 
>>
>>  (list (thing-at-point 'line) (current-buffer) (point))
>>
>> doesn't cost anything and would go a long way to give a better error
>> message.  If thing-at-point returns nil or an empty line then
>> identifier-at-point should just return nil.
>
> Bearing in mind the example I mentioned in the previous email, maybe we
> can add an optional argument to the xref-identifier-at-point-function.

You could also further restrict the identifier type: allow only strings.
Tell users that they can put text properties on the string if they need
more structured data.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/15/2014 11:57 PM, Helmut Eller wrote:

> You could also further restrict the identifier type: allow only strings.

Already did. :) Have you looked at the patch?

> Tell users that they can put text properties on the string if they need
> more structured data.

Right, that will also be a nice escape hatch. Do you think it obviates 
my proposal of returning precise or imprecise identifiers, depending on 
the argument?

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Dec 16 2014, Dmitry Gutov wrote:

> On 12/15/2014 11:57 PM, Helmut Eller wrote:
>
>> You could also further restrict the identifier type: allow only strings.
>
> Already did. :) Have you looked at the patch?

Apparently I missed that.  The last patch I looked at had a number of
(eq id t) tests.

>> Tell users that they can put text properties on the string if they need
>> more structured data.
>
> Right, that will also be a nice escape hatch. Do you think it obviates
> my proposal of returning precise or imprecise identifiers, depending
> on the argument?

There's still the problem of how to put text properties on the result of
completing-read.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/16/2014 12:17 AM, Helmut Eller wrote:

> Apparently I missed that.  The last patch I looked at had a number of
> (eq id t) tests.

Ok, sorry. I misunderstood. So, only strings? That's not bad, the string 
can be that "imprecise" value, and a property will signal that the 
backend will have to find out what the identifier is, maybe using the 
value of that property (likely marker), or the current value of point.

> There's still the problem of how to put text properties on the result of
> completing-read.

Do we really need to? I think the identifier-completion-table can manage 
to contain only unambiguous entries.

Even if we could carry text properties through `completing-read', the 
user won't see them when choosing, so all strings in the table will have 
to be different anyway.

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Dec 16 2014, Dmitry Gutov wrote:

> Ok, sorry. I misunderstood. So, only strings? That's not bad, the
> string can be that "imprecise" value, and a property will signal that
> the backend will have to find out what the identifier is, maybe using
> the value of that property (likely marker), or the current value of
> point.

Yes, that's the idea.

>> There's still the problem of how to put text properties on the result of
>> completing-read.
>
> Do we really need to? I think the identifier-completion-table can
> manage to contain only unambiguous entries.
>
> Even if we could carry text properties through `completing-read', the
> user won't see them when choosing, so all strings in the table will
> have to be different anyway.

What I'm saying is that my original proposal of having a
read-identifier-form-minibuffer make sense as it allows backends to add
text properties more easily than a completion table.  And calling
completing-read isn't difficult for the backends if they already have
the completion table.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/16/2014 12:41 AM, Helmut Eller wrote:
>> Even if we could carry text properties through `completing-read', the
>> user won't see them when choosing, so all strings in the table will
>> have to be different anyway.
>
> What I'm saying is that my original proposal of having a
> read-identifier-form-minibuffer make sense as it allows backends to add
> text properties more easily than a completion table.

And I'm asking, why do we need text properties there?

For example, for a hypothetical Ruby backend, the completion table will 
return fully qualified identifiers, like String#gsub, 
ActiveRecord::Base.first, or Hash#[]. A string like this can be send to 
the external process, and it'll return the source location, docstring, 
etc, without any additional data.

 > And calling
> completing-read isn't difficult for the backends if they already have
> the completion table.

That's a lot of responsibility we don't necessarily have to give them. 
And even though the method is called `read-identifier-from-minibuffer', 
what's stopping them from using any other possible interface to get the 
input from the user?

Rather than allow that, I'd rather we improve the input interface in 
centralized fashion, and ask the backends only for possible values.

So when we can read input in the middle of the frame in a nice popup 
(any day now, I'm sure), all backends will benefit.

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Dec 16 2014, Dmitry Gutov wrote:

> On 12/16/2014 12:41 AM, Helmut Eller wrote:
>>> Even if we could carry text properties through `completing-read', the
>>> user won't see them when choosing, so all strings in the table will
>>> have to be different anyway.
>>
>> What I'm saying is that my original proposal of having a
>> read-identifier-form-minibuffer make sense as it allows backends to add
>> text properties more easily than a completion table.
>
> And I'm asking, why do we need text properties there?

Because it feels right.

>> And calling
>> completing-read isn't difficult for the backends if they already have
>> the completion table.
>
> That's a lot of responsibility we don't necessarily have to give
> them. And even though the method is called
> `read-identifier-from-minibuffer', what's stopping them from using any
> other possible interface to get the input from the user?
>
> Rather than allow that, I'd rather we improve the input interface in
> centralized fashion, and ask the backends only for possible values.

>
> So when we can read input in the middle of the frame in a nice popup
> (any day now, I'm sure), all backends will benefit.

Feel free to disagree.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/16/2014 10:42 PM, Helmut Eller wrote:
> Not everything needs to be entered by the user.  The namespace part
> could comes from the buffer (inferred)

But what if they want to jump to a symbol defined in a different 
package/namespace?

I would imagine that jumping to a symbol in the current package would be 
relatively infrequent (in one-file-per-class languages), or at least not 
as important (in other languages, where Imenu probably covers that).

Admittedly, if the language requires all used external symbols (classes, 
or functions) to be declared at the top of the file, for all symbols 
mentioned in the current file "jump to definition" could just be based 
on the local name (maybe imported alias) and the name of the current 
unit of computation (package or class, like in Java), but that discounts 
all symbols that aren't referenced in the current file.

I think allowing to jump to an arbitrary symbol (not just the one near 
point, or even in the same package) is the main goal for the identifier 
completion table.

 > and the user shouldn't be forced to type fully qualified names.

Ideally, right. We want to offer the user defaults for both the package 
part (current), and the symbol name (symbol at point), both of which the 
user could override conveniently.

But maybe offering current.package/symbol-at-point as one default value, 
in one completing-read invocation, would be good enough?

The benefit of a backend being able to decide to call `completing-read' 
twice is being able to cleanly offer these two defaults, separately.

The drawback: personally, I can't really choose which part to prompt for 
first. The best result probably depends on whether the user is looking 
for a symbol (and isn't sure about the package), or knows the package 
and wants to explore the symbols it it.

If we're completing both a the same time, the user can choose which part 
to edit, with completion. I'm not sure, though, if the backend would be 
able to actually filter completions for the first part (package) based 
on the text of the second part (symbol name) thus far entered. But 
that's an implementation detail.

To recount, here are arguments in favor of having identifier completion 
table as a part of the backend API, from the older discussion:

- Partial-completion style will let you complete "js.fo" to anything 
that matches "js*.fo*".

- It means the UI can use it for various kinds of completion (e.g. for
completion-at-point-function).

- ...the general rule that the backend should not do UI tasks and 
prompting is definitely a UI operation.

Re: Generalizing find-definition

[Helmut Eller]

On Tue, Dec 16 2014, Dmitry Gutov wrote:

2> On 12/16/2014 10:42 PM, Helmut Eller wrote:
>> Not everything needs to be entered by the user.  The namespace part
>> could comes from the buffer (inferred)
>
> But what if they want to jump to a symbol defined in a different
> package/namespace?

Then there's no other choice than to use a qualified name.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

Helmut Eller <eller.helmut@gmail.com> writes:

>> But what if they want to jump to a symbol defined in a different
>> package/namespace?
>
> Then there's no other choice than to use a qualified name.

Okay.

Anyway, the identifier table can just as well include local names (and
aliases), all of which `xref-find-function' should be able to resolve
with no problem because it's called in the same buffer.

And strings in completion tables can include properties, too.

I've pushed the patch with some updates to
http://git.savannah.gnu.org/cgit/emacs.git/log/?h=scratch/xref, please
everyone take a look.

If there are no further objections, I'll install it in a few days.

Re: Generalizing find-definition

[Helmut Eller]

On Fri, Dec 19 2014, Dmitry Gutov wrote:

> I've pushed the patch with some updates to
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=scratch/xref, please
> everyone take a look.

xref-goto-xref should be rewritten as:

(defun xref-goto-xref ()
  "Jump to the xref at point and close the xref buffer."
  (interactive)
  (let ((loc (xref--location-at-point)))
    (quit-window)
    (xref--pop-to-location loc)))

as the old version could quit the wrong window.

I would also add this to make the next-error command work:

(defun xref--next-error-function (n reset?)
  (when reset?
    (goto-char (point-min)))
  (let ((backward (< n 0))
	(n (abs n))
	(loc nil))
    (dotimes (_ n)
      (setq loc (xref--search-property 'xref-location backward)))
    (cond (loc
	   (xref--display-position (point) t 0)
	   (xref--pop-to-location loc))
	  (t
	   (error "No %s xref" (if backward "previous" "next"))))))

(define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
  "Mode for displaying cross refenences."
  (setq buffer-read-only t)
  (setq next-error-function #'xref--next-error-function)
  (setq next-error-last-buffer (current-buffer)))

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/19/2014 10:49 AM, Helmut Eller wrote:

 > xref-goto-xref should be rewritten as:

Ok, thanks.

> (define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
>    "Mode for displaying cross refenences."
>    (setq buffer-read-only t)
>    (setq next-error-function #'xref--next-error-function)
>    (setq next-error-last-buffer (current-buffer)))

This looks dubious to me. What if I already have a compilation buffer 
open, and I'm using `xref-find-definitions' to follow the implementation 
of some code near one of the errors?

This seems like it'll break the error chain I'm currently following.

Also, when I'm using xref-find-definitions (as opposed to the other 
xref-find- functions), I'm very unlikely to want to visit all results. 
It's usually just one that I'm after.

Re: Generalizing find-definition

[Helmut Eller]

On Fri, Dec 19 2014, Dmitry Gutov wrote:

> I've pushed the patch with some updates to
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=scratch/xref, please
> everyone take a look.

in elisp-mode.el I would use

(eval-when-compile
  (require 'xref))

instead of declare-function.

Helmut

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/19/2014 10:56 AM, Helmut Eller wrote:

> (eval-when-compile
>    (require 'xref))
>
> instead of declare-function.

If I do that, the byte-compiler still complains that these functions 
"might not be defined at runtime".

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/19/2014 10:00 AM, Dmitry Gutov wrote:

> If there are no further objections, I'll install it in a few days.

Installed, with minor tweaks.

I'll let others decide if the now-unbound interactive functions from 
etags.el should be obsoleted, removed or kept as-is for older users.

Re: Generalizing find-definition

[Stefan Monnier]

> I'll let others decide if the now-unbound interactive functions from
> etags.el should be obsoleted, removed or kept as-is for older users.

Obsoleting them sounds right,


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/26/2014 05:50 AM, Stefan Monnier wrote:

> Obsoleting them sounds right,

How about this? Maybe install this early and then collect complaints if 
the new system is not up to par.


diff --git a/etc/NEWS b/etc/NEWS
index 14933aa..306e840 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -453,6 +453,11 @@ easier binding, which is now unoccupied (`M-,').
  alias for a private variable.  `xref-push-marker-stack' and
  `xref-pop-marker-stack' should be used to mutate it instead.

+** etags
+As a result of the above, these commands are now obsolete:
+`find-tag-other-window', `find-tag-other-frame', `find-tag-regexp',
+`tags-apropos' and `tags-loop-continue'.
+
  ** Obsolete packages

  ---
diff --git a/lisp/menu-bar.el b/lisp/menu-bar.el
index 8f33641..9eedc0a 100644
--- a/lisp/menu-bar.el
+++ b/lisp/menu-bar.el
@@ -379,29 +379,18 @@
        menu-bar-separator)

      (bindings--define-key menu [apropos-tags]
-      '(menu-item "Tags Apropos..." tags-apropos
+      '(menu-item "Find Apropos..." xref-find-apropos
                    :help "Find function/variables whose names match 
regexp"))
-    (bindings--define-key menu [next-tag-otherw]
-      '(menu-item "Next Tag in Other Window"
-                  menu-bar-next-tag-other-window
-                  :enable (and (boundp 'tags-location-ring)
-                               (not (ring-empty-p tags-location-ring)))
-                  :help "Find next function/variable matching last tag 
name in another window"))
-
-    (bindings--define-key menu [next-tag]
-      '(menu-item "Find Next Tag"
-                  menu-bar-next-tag
-                  :enable (and (boundp 'tags-location-ring)
-                               (not (ring-empty-p tags-location-ring)))
-                  :help "Find next function/variable matching last tag 
name"))
-    (bindings--define-key menu [find-tag-otherw]
-      '(menu-item "Find Tag in Other Window..." find-tag-other-window
-                  :help "Find function/variable definition in another 
window"))
-    (bindings--define-key menu [find-tag]
-      '(menu-item "Find Tag..." find-tag
-                  :help "Find definition of function or variable"))
-
-    (bindings--define-key menu [separator-tags]
+
+    (bindings--define-key menu [xref-find-otherw]
+      '(menu-item "Find Definition in Other Window..."
+                  xref-find-definitions-other-window
+                  :help "Find function/variable definitions in another 
window"))
+    (bindings--define-key menu [xref-find-def]
+      '(menu-item "Find Definition..." xref-find-definitions
+                  :help "Find definitions of function or variable"))
+
+    (bindings--define-key menu [separator-xref]
        menu-bar-separator)

      (bindings--define-key menu [end-of-buf]
diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index 0be9979..f9e037b 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -995,6 +995,9 @@ See documentation of variable `tags-file-name'."
  			(set-window-point (selected-window) tagpoint))
  		      window-point)))

+(make-obsolete 'find-tag-other-window
+               'xref-find-definitions-other-window "25.1")
+
  ;;;###autoload
  (defun find-tag-other-frame (tagname &optional next-p)
    "Find tag (in current tags table) whose name contains TAGNAME.
@@ -1019,6 +1022,9 @@ See documentation of variable `tags-file-name'."
    (let ((pop-up-frames t))
      (find-tag-other-window tagname next-p)))

+(make-obsolete 'find-tag-other-frame
+               'xref-find-definitions-other-frame "25.1")
+
  ;;;###autoload
  (defun find-tag-regexp (regexp &optional next-p other-window)
    "Find tag (in current tags table) whose name matches REGEXP.
@@ -1042,6 +1048,8 @@ See documentation of variable `tags-file-name'."
    (funcall (if other-window 'find-tag-other-window 'find-tag)
  	   regexp next-p t))

+(make-obsolete 'find-tag-regexp 'xref-find-apropos "25.1")
+
  ;;;###autoload
  (defalias 'pop-tag-mark 'xref-pop-marker-stack)

@@ -1842,6 +1850,10 @@ nil, we exit; otherwise we scan the next file."
  	 (null tags-loop-operate)
  	 (message "Scanning file %s...found" buffer-file-name))))

+(make-obsolete 'tags-loop-continue
+               "use `xref-find-definitions' interface instead."
+               "25.1")
+
  ;;;###autoload
  (defun tags-search (regexp &optional file-list-form)
    "Search through all files listed in tags table for match for REGEXP.
@@ -1946,6 +1958,9 @@ directory specification."
      ;; apropos-mode is derived from fundamental-mode and it kills
      ;; all local variables.
      (setq buffer-read-only t)))
+
+(make-obsolete 'tags-apropos 'xref-find-apropos "25.1")
+
  \f
  ;; XXX Kludge interface.

Re: Generalizing find-definition

[Stefan Monnier]

> How about this? Maybe install this early and then collect complaints if the
> new system is not up to par.

Please use (declare (obsolete ...)) instead.


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/29/2014 02:24 AM, Stefan Monnier wrote:

> Please use (declare (obsolete ...)) instead.

Right!

Thanks for reading, installed.

Re: Generalizing find-definition

[Dmitry Gutov]

Can we remove the commands `menu-bar-next-tag-other-window' and 
`menu-bar-next-tag'?

Re: Generalizing find-definition

[Stefan Monnier]

> Can we remove the commands `menu-bar-next-tag-other-window' and
> `menu-bar-next-tag'?

Yes, go ahead.


        Stefan

Re: Generalizing find-definition

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 29 Dec 2014 09:20:30 -0500
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> > Can we remove the commands `menu-bar-next-tag-other-window' and
> > `menu-bar-next-tag'?
> 
> Yes, go ahead.

Why???

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/29/2014 06:17 PM, Eli Zaretskii wrote:

>>> Can we remove the commands `menu-bar-next-tag-other-window' and
>>> `menu-bar-next-tag'?
>>
>> Yes, go ahead.
>
> Why???

Because the menu elements had been removed, and these seem like they 
only should be used in the menu (even though the names look public).

Re: Generalizing find-definition

[Eli Zaretskii]

> Date: Mon, 29 Dec 2014 19:27:10 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: emacs-devel@gnu.org
> 
> On 12/29/2014 06:17 PM, Eli Zaretskii wrote:
> 
> >>> Can we remove the commands `menu-bar-next-tag-other-window' and
> >>> `menu-bar-next-tag'?
> >>
> >> Yes, go ahead.
> >
> > Why???
> 
> Because the menu elements had been removed, and these seem like they 
> only should be used in the menu (even though the names look public).

Sorry, I don't follow: which menu elements were removed?

Re: Generalizing find-definition

[Stefan Monnier]

>> > Can we remove the commands `menu-bar-next-tag-other-window' and
>> > `menu-bar-next-tag'?
>> Yes, go ahead.
> Why???

They're not used any more.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Dmitry Gutov <raaahh@gmail.com> writes:

> On 12/19/2014 10:00 AM, Dmitry Gutov wrote:
>
>> If there are no further objections, I'll install it in a few days.
>
> Installed, with minor tweaks.
>
> I'll let others decide if the now-unbound interactive functions from
> etags.el should be obsoleted, removed or kept as-is for older users.

Something's weird.

Starting from emacs -Q, in *scratch*, eval:

(require 'xref)
(emacs-lisp-mode)
xref-find-function

This gives etags-xref-find; it should give elisp-xref-find.

The only work-around I've found is to recompile emacs-lisp-mode in the
running emacs.

Somehow, the dumped definition of emacs-lisp-mode doesn't match the
source in elisp-mode.el (which sets xref-find-function to
elisp-xref-find).

-- 
-- Stephe

Re: Generalizing find-definition

[Stephen Leake]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Dmitry Gutov <raaahh@gmail.com> writes:
>
>> On 12/19/2014 10:00 AM, Dmitry Gutov wrote:
>>
>>> If there are no further objections, I'll install it in a few days.
>>
>> Installed, with minor tweaks.
>>
>> I'll let others decide if the now-unbound interactive functions from
>> etags.el should be obsoleted, removed or kept as-is for older users.
>
> Something's weird.
>
> Starting from emacs -Q, in *scratch*, eval:
>
> (require 'xref)
> (emacs-lisp-mode)
> xref-find-function
>
> This gives etags-xref-find; it should give elisp-xref-find.
>
> The only work-around I've found is to recompile emacs-lisp-mode in the
> running emacs.
>
> Somehow, the dumped definition of emacs-lisp-mode doesn't match the
> source in elisp-mode.el (which sets xref-find-function to
> elisp-xref-find).

Never mind; I forgot to recompile in that build dir.

-- 
-- Stephe

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:


> - identifier-at-point should document more clearly that the return value
>   may be something else than a string.  AFAIK the only interesting
>   non-string non-nil case is to return a special value (e.g. a marker
>   or just `t' to stand to (point)) which stands for "let the
>   find-definitions code figure out the identifier at that place".

In ada-mode, identifier is a class derived from xref-file-location:

(defclass xref-ada-identifier (xref-file-location)
  ((id :type string :initarg :identifier)))

This is because identifiers in Ada are overloaded, and are often just
the last part of the fully qualified namespace name. The backend uses
the cross-reference information output by the compiler; the compiler
does the name resolution.

If identifier is restricted to just the string, ada-mode needs a
guarantee that xref-find-definitions will be called with current-buffer
the same as the buffer containing the string, and point in the same
place, so the backend can determine the file, line, column. That seems
kludgy.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> If identifier is restricted to just the string, ada-mode needs a
> guarantee that xref-find-definitions will be called with current-buffer
> the same as the buffer containing the string, and point in the same
> place, so the backend can determine the file, line, column.

That's the idea behind using the value t as an "identifier at point".
It seems like a reasonable constraint on the use of
xref-find-definitions.  But if you can think of a case where it might be
impractical to call xref-find-definitions from the right place, we could
use a marker instead of the value t.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> If identifier is restricted to just the string, ada-mode needs a
>> guarantee that xref-find-definitions will be called with current-buffer
>> the same as the buffer containing the string, and point in the same
>> place, so the backend can determine the file, line, column.
>
> That's the idea behind using the value t as an "identifier at point".
> It seems like a reasonable constraint on the use of
> xref-find-definitions.  But if you can think of a case where it might be
> impractical to call xref-find-definitions from the right place, we could
> use a marker instead of the value t.

I could live with either `t' or a marker; we can make it more
complicated later if some mode shows up with a real need. A marker is
clearer than just `t', but someone has to be responsible for deleting
the marker.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> I could live with either `t' or a marker; we can make it more
> complicated later if some mode shows up with a real need. A marker is
> clearer than just `t', but someone has to be responsible for deleting
> the marker.

No, markers are automatically "deleted" by the GC (contrary to overlays).


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 1301 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> That's the idea behind using the value t as an "identifier at point".
> It seems like a reasonable constraint on the use of
> xref-find-definitions.

Here's another example why we could reconsider using simply t or marker
as a return value there:

- Suppose, for the current backend, xref-identifier-at-point-function
  never returns strings, as described.

- User calls `xref-find-definitions' with a prefix argument, this
  triggers completing read, but we can't use anything as the initial
  input.

For a popular OOP language, the identifier completion table might
contain entries like "package.Class#method", and if the point is on
"foo.method", and if completion doesn't require prefix matching, having
"method" as the initial input would save the user some typing.

Anyway, updated patch attached. Aside from what was already discussed,
the changes are:

- Added `xref-find-apropos' and its implementation in the Etags backend,
  as a replacement for reading and jumping to a "regexp identifier".

- `xref-identifier-completion-table' ->
  `xref-identifier-table-function'. Apparently, it should be a function
  returning a completion table, for the same reason as why
  `tags-lazy-completion-table' exists. Please correct me if I'm wrong.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Plain function API with EIEIO locations --]
[-- Type: text/x-diff, Size: 27999 bytes --]

diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ba70f90..b051605 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -227,10 +227,14 @@ Blank lines separate paragraphs.  Semicolons start comments.
 
 \\{emacs-lisp-mode-map}"
   :group 'lisp
+  (defvar xref-find-function)
+  (defvar xref-identifier-table-function)
   (lisp-mode-variables nil nil 'elisp)
   (setq imenu-case-fold-search nil)
   (setq-local eldoc-documentation-function
               #'elisp-eldoc-documentation-function)
+  (setq-local xref-find-function #'elisp-xref-find)
+  (setq-local xref-identifier-table-function #'elisp--xref-identifier-table)
   (add-hook 'completion-at-point-functions
             #'elisp-completion-at-point nil 'local))
 
@@ -426,6 +430,15 @@ It can be quoted, or be inside a quoted form."
             0))
      ((facep sym) (find-definition-noselect sym 'defface)))))
 
+(defvar elisp--identifier-completion-table
+  (apply-partially #'completion-table-with-predicate
+                   obarray
+                   (lambda (sym)
+                     (or (boundp sym)
+                         (fboundp sym)
+                         (symbol-plist sym)))
+                   'strict))
+
 (defun elisp-completion-at-point ()
   "Function used for `completion-at-point-functions' in `emacs-lisp-mode'."
   (with-syntax-table emacs-lisp-mode-syntax-table
@@ -466,13 +479,8 @@ It can be quoted, or be inside a quoted form."
                            :company-docsig #'elisp--company-doc-string
                            :company-location #'elisp--company-location))
                     ((elisp--form-quoted-p beg)
-                     (list nil obarray
-                           ;; Don't include all symbols
-                           ;; (bug#16646).
-                           :predicate (lambda (sym)
-                                        (or (boundp sym)
-                                            (fboundp sym)
-                                            (symbol-plist sym)))
+                     ;; Don't include all symbols (bug#16646).
+                     (list nil elisp--identifier-completion-table
                            :annotation-function
                            (lambda (str) (if (fboundp (intern-soft str)) " <f>"))
                            :company-doc-buffer #'elisp--company-doc-buffer
@@ -548,6 +556,34 @@ It can be quoted, or be inside a quoted form."
 (define-obsolete-function-alias
   'lisp-completion-at-point 'elisp-completion-at-point "25.1")
 
+;;; Xref backend
+
+(declare-function xref-make-buffer-location "xref" (buffer position))
+(declare-function xref-make-bogus-location "xref" (message))
+(declare-function xref-make "xref" (description location))
+
+;; FIXME: unify with `elisp--company-location'.
+(defun elisp-xref-find (action id)
+  (when (eq action 'definitions)
+    (let ((sym (intern-soft id)))
+      (when sym
+        (let ((fun (if (fboundp sym) (elisp--xref-find-type sym nil)))
+              (var (if (boundp sym) (elisp--xref-find-type sym 'defvar))))
+          (remove nil (list fun var)))))))
+
+(defun elisp--xref-find-type (symbol type)
+  (let ((loc (condition-case err
+                 (let ((loc (save-excursion
+                              (find-definition-noselect symbol type))))
+                   (xref-make-buffer-location (car loc) (or (cdr loc) 1)))
+               (error
+                (xref-make-bogus-location (error-message-string err)))))
+        (desc (format "(%s %s)" (or type 'defun) symbol)))
+    (xref-make desc loc)))
+
+(defun elisp--xref-identifier-table ()
+  elisp--identifier-completion-table)
+
 ;;; Elisp Interaction mode
 
 (defvar lisp-interaction-mode-map
diff --git a/lisp/progmodes/etags.el b/lisp/progmodes/etags.el
index b89b4cf..37c14a9 100644
--- a/lisp/progmodes/etags.el
+++ b/lisp/progmodes/etags.el
@@ -28,6 +28,7 @@
 
 (require 'ring)
 (require 'button)
+(require 'xref)
 
 ;;;###autoload
 (defvar tags-file-name nil
@@ -182,8 +183,8 @@ Example value:
 		       (sexp :tag "Tags to search")))
   :version "21.1")
 
-(defvar find-tag-marker-ring (make-ring find-tag-marker-ring-length)
-  "Ring of markers which are locations from which \\[find-tag] was invoked.")
+(define-obsolete-variable-alias 'find-tag-marker-ring 'xref--marker-ring
+  "25.1")
 
 (defvar default-tags-table-function nil
   "If non-nil, a function to choose a default tags file for a buffer.
@@ -716,12 +717,10 @@ Returns t if it visits a tags table, or nil if there are no more in the list."
     (while (< i find-tag-marker-ring-length)
       (if (aref (cddr tags-location-ring) i)
 	  (set-marker (aref (cddr tags-location-ring) i) nil))
-      (if (aref (cddr find-tag-marker-ring) i)
-	  (set-marker (aref (cddr find-tag-marker-ring) i) nil))
       (setq i (1+ i))))
+  (xref-clear-marker-stack)
   (setq tags-file-name nil
 	tags-location-ring (make-ring find-tag-marker-ring-length)
-	find-tag-marker-ring (make-ring find-tag-marker-ring-length)
 	tags-table-list nil
 	tags-table-computed-list nil
 	tags-table-computed-list-for nil
@@ -780,6 +779,7 @@ tags table and its (recursively) included tags tables."
 	(quit (message "Tags completion table construction aborted.")
 	      (setq tags-completion-table nil)))))
 
+;;;###autoload
 (defun tags-lazy-completion-table ()
   (let ((buf (current-buffer)))
     (lambda (string pred action)
@@ -898,7 +898,7 @@ See documentation of variable `tags-file-name'."
 	      ;; Run the user's hook.  Do we really want to do this for pop?
 	      (run-hooks 'local-find-tag-hook))))
       ;; Record whence we came.
-      (ring-insert find-tag-marker-ring (point-marker))
+      (xref-push-marker-stack)
       (if (and next-p last-tag)
 	  ;; Find the same table we last used.
 	  (visit-tags-table-buffer 'same)
@@ -954,7 +954,6 @@ See documentation of variable `tags-file-name'."
 	(switch-to-buffer buf)
       (error (pop-to-buffer buf)))
     (goto-char pos)))
-;;;###autoload (define-key esc-map "." 'find-tag)
 
 ;;;###autoload
 (defun find-tag-other-window (tagname &optional next-p regexp-p)
@@ -995,7 +994,6 @@ See documentation of variable `tags-file-name'."
 			;; the window's point from the buffer.
 			(set-window-point (selected-window) tagpoint))
 		      window-point)))
-;;;###autoload (define-key ctl-x-4-map "." 'find-tag-other-window)
 
 ;;;###autoload
 (defun find-tag-other-frame (tagname &optional next-p)
@@ -1020,7 +1018,6 @@ See documentation of variable `tags-file-name'."
   (interactive (find-tag-interactive "Find tag other frame: "))
   (let ((pop-up-frames t))
     (find-tag-other-window tagname next-p)))
-;;;###autoload (define-key ctl-x-5-map "." 'find-tag-other-frame)
 
 ;;;###autoload
 (defun find-tag-regexp (regexp &optional next-p other-window)
@@ -1049,20 +1046,8 @@ See documentation of variable `tags-file-name'."
 ;;;###autoload (define-key esc-map "*" 'pop-tag-mark)
 
 ;;;###autoload
-(defun pop-tag-mark ()
-  "Pop back to where \\[find-tag] was last invoked.
+(defalias 'pop-tag-mark 'xref-pop-marker-stack)
 
-This is distinct from invoking \\[find-tag] with a negative argument
-since that pops a stack of markers at which tags were found, not from
-where they were found."
-  (interactive)
-  (if (ring-empty-p find-tag-marker-ring)
-      (error "No previous locations for find-tag invocation"))
-  (let ((marker (ring-remove find-tag-marker-ring 0)))
-    (switch-to-buffer (or (marker-buffer marker)
-                          (error "The marked buffer has been deleted")))
-    (goto-char (marker-position marker))
-    (set-marker marker nil nil)))
 \f
 (defvar tag-lines-already-matched nil
   "Matches remembered between calls.") ; Doc string: calls to what?
@@ -1859,7 +1844,6 @@ nil, we exit; otherwise we scan the next file."
     (and messaged
 	 (null tags-loop-operate)
 	 (message "Scanning file %s...found" buffer-file-name))))
-;;;###autoload (define-key esc-map "," 'tags-loop-continue)
 
 ;;;###autoload
 (defun tags-search (regexp &optional file-list-form)
@@ -2077,6 +2061,54 @@ for \\[find-tag] (which see)."
       (completion-in-region (car comp-data) (cadr comp-data)
 			    (nth 2 comp-data)
 			    (plist-get (nthcdr 3 comp-data) :predicate)))))
+
+\f
+;;; Xref backend
+
+;; Stop searching if we find more than xref-limit matches, as the xref
+;; infrastracture is not designed to handle very long lists.
+;; Switching to some kind of lazy list might be better, but hopefully
+;; we hit the limit rarely.
+(defconst etags--xref-limit 1000)
+
+;;;###autoload
+(defun etags-xref-find (action id)
+  (pcase action
+    (`definitions (etags--xref-find-definitions id))
+    (`apropos (etags--xref-find-definitions id t))))
+
+(defun etags--xref-find-definitions (pattern &optional regexp?)
+  ;; This emulates the behaviour of `find-tag-in-order' but instead of
+  ;; returning one match at a time all matches are returned as list.
+  ;; NOTE: find-tag-tag-order is typically a buffer-local variable.
+  (let* ((xrefs '())
+         (first-time t)
+         (search-fun (if regexp? #'re-search-forward #'search-forward))
+         (marks (make-hash-table :test 'equal))
+         (case-fold-search (if (memq tags-case-fold-search '(nil t))
+                               tags-case-fold-search
+                             case-fold-search)))
+    (save-excursion
+      (while (visit-tags-table-buffer (not first-time))
+        (setq first-time nil)
+        (dolist (order-fun (cond (regexp? find-tag-regexp-tag-order)
+                                 (t find-tag-tag-order)))
+          (goto-char (point-min))
+          (while (and (funcall search-fun pattern nil t)
+                      (< (hash-table-count marks) etags--xref-limit))
+            (when (funcall order-fun pattern)
+              (beginning-of-line)
+              (cl-destructuring-bind (hint line &rest pos) (etags-snarf-tag)
+                (unless (eq hint t) ; hint==t if we are in a filename line
+                  (let* ((file (file-of-tag))
+                         (mark-key (cons file line)))
+                    (unless (gethash mark-key marks)
+                      (let ((loc (xref-make-file-location
+                                  (expand-file-name file) line 0)))
+                        (push (xref-make hint loc) xrefs)
+                        (puthash mark-key t marks)))))))))))
+    (nreverse xrefs)))
+
 \f
 (provide 'etags)
 
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
new file mode 100644
index 0000000..780c586
--- /dev/null
+++ b/lisp/progmodes/xref.el
@@ -0,0 +1,477 @@
+;; xref.el --- Cross-referencing commands              -*-lexical-binding:t-*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; This file provides a somewhat generic infrastructure for cross
+;; referencing commands, in particular "find-definition".
+;;
+;; Some part of the functionality must be implemented in a language
+;; dependent way and that's done by defining `xref-find-function',
+;; `xref-identifier-at-point-function' and
+;; `xref-identifier-table-function', which see.  A language-specific
+;; mode usually makes these variables buffer-local first.
+;;
+;; For `xref-find-function' calling conventions, see its description.
+;; It has to operate with "xref" and "location" values.
+;;
+;; One would usually call `make-xref' and `xref-make-file-location',
+;; `xref-make-buffer-location' or `xref-make-bogus-location' to create
+;; them.
+;;
+;; For each identifier, we consider that either it has a precise
+;; string representation that's easy to find out (in which case we
+;; operate with a string value), or we use the value t if a background
+;; process is expected to determine it using the buffer contents and
+;; the current position.
+;;
+;; See the functions `etags-xref-find' and `elisp-xref-find' for full
+;; examples.
+
+;;; Code:
+
+(require 'cl-lib)
+(require 'eieio)
+(require 'ring)
+
+\f
+;;; Locations
+
+(defclass xref-location () ()
+  :documentation "A location represents a position in a file or buffer.")
+
+;; If a backend decides to subclass xref-location it can provide
+;; methods for some of the following functions:
+(defgeneric xref-location-marker (location)
+  "Return the marker for LOCATION.")
+
+(defgeneric xref-location-group (location)
+  "Return a string used to group a set of locations.
+This is typically the filename.")
+
+;;;; Commonly needed location classes are defined here:
+
+;; FIXME: might be useful to have an optional "hint" i.e. a string to
+;; search for in case the line number is sightly out of date.
+(defclass xref-file-location (xref-location)
+  ((file :type string :initarg :file)
+   (line :type fixnum :initarg :line)
+   (column :type fixnum :initarg :column))
+  :documentation "A file location is a file/line/column triple.
+Line numbers start from 1 and columns from 0.")
+
+(defun xref-make-file-location (file line column)
+  "Create and return a new xref-file-location."
+  (make-instance 'xref-file-location :file file :line line :column column))
+
+(defmethod xref-location-marker ((l xref-file-location))
+  (with-slots (file line column) l
+    (with-current-buffer
+        (or (get-file-buffer file)
+            (let ((find-file-suppress-same-file-warnings t))
+              (find-file-noselect file)))
+      (save-restriction
+        (widen)
+        (save-excursion
+          (goto-char (point-min))
+          (beginning-of-line line)
+          (move-to-column column)
+          (point-marker))))))
+
+(defmethod xref-location-group ((l xref-file-location))
+  (oref l :file))
+
+(defclass xref-buffer-location (xref-location)
+  ((buffer :type buffer :initarg :buffer)
+   (position :type fixnum :initarg :position)))
+
+(defun xref-make-buffer-location (buffer position)
+  "Create and return a new xref-buffer-location."
+  (make-instance 'xref-buffer-location :buffer buffer :position position))
+
+(defmethod xref-location-marker ((l xref-buffer-location))
+  (with-slots (buffer position) l
+    (let ((m (make-marker)))
+      (move-marker m position buffer))))
+
+(defmethod xref-location-group ((l xref-buffer-location))
+  (with-slots (buffer) l
+    (or (buffer-file-name buffer)
+        (format "(buffer %s)" (buffer-name buffer)))))
+
+(defclass xref-bogus-location (xref-location)
+  ((message :type string :initarg :message
+            :reader xref-bogus-location-message))
+  :documentation "Bogus locations are sometimes useful to
+indicate errors, e.g. when we know that a function exists but the
+actual location is not known.")
+
+(defun xref-make-bogus-location (message)
+  "Create and return a new xref-bogus-location."
+  (make-instance 'xref-bogus-location :message message))
+
+(defmethod xref-location-marker ((l xref-bogus-location))
+  (user-error "%s" (oref l :message)))
+
+(defmethod xref-location-group ((_ xref-bogus-location)) "(No location)")
+
+\f
+;;; Cross-reference
+
+(defclass xref--xref ()
+  ((description :type string :initarg :description
+                :reader xref--xref-description)
+   (location :type xref-location :initarg :location
+             :reader xref--xref-location))
+  :comment "An xref is used to display and locate constructs like
+variables or functions.")
+
+(defun xref-make (description location)
+  "Create and return a new xref.
+DESCRIPTION is a short string to describe the xref.
+LOCATION is an `xref-location'."
+  (make-instance 'xref--xref :description description :location location))
+
+\f
+;;; API
+
+;; For now, make the etags backend the default.
+(defvar xref-find-function #'etags-xref-find
+  "Function to look for cross-references.
+It can be called in several ways:
+
+ (definitions IDENTIFIER): Find definitions of IDENTIFIER.  The
+result must be a list of xref objects.  If no definitions can be
+found, return nil.
+
+ (references IDENTIFIER): Find references of IDENTIFIER.  The
+result must be a list of xref objects.  If no references can be
+found, return nil.
+
+ (apropos PATTERN): Find all symbols that match PATTERN.  PATTERN
+is a regexp.
+
+IDENTIFIER can be any non-nil value returned by
+`xref-identifier-at-point-function', or any value in
+`xref-identifier-table-function'.
+
+To create an xref object, call `xref-make'.")
+
+(defvar xref-identifier-at-point-function #'xref-default-identifier-at-point
+  "Function to get the relevant identifier at point.
+
+The return value must be a string, t or nil.  nil means no
+identifier at point found.  t means that there is an identifier
+at point, but its string representation is difficult to obtain.")
+
+(defvar xref-identifier-table-function #'tags-lazy-completion-table
+  "Function that returns the completion table for identifiers.")
+
+(defun xref-default-identifier-at-point ()
+  (let ((thing (thing-at-point 'symbol)))
+    (and thing (substring-no-properties thing))))
+
+\f
+;;; misc utilities
+(defun xref--alistify (list key test)
+  "Partition the elements of LIST into an alist.
+KEY extracts the key from an element and TEST is used to compare
+keys."
+  (let ((alist '()))
+    (dolist (e list)
+      (let* ((k (funcall key e))
+             (probe (cl-assoc k alist :test test)))
+        (if probe
+            (setcdr probe (cons e (cdr probe)))
+          (push (cons k (list e)) alist))))
+    ;; Put them back in order.
+    (cl-loop for (key . value) in (reverse alist)
+             collect (cons key (reverse value)))))
+
+(defun xref--insert-propertized (props &rest strings)
+  "Insert STRINGS with text properties PROPS."
+  (let ((start (point)))
+    (apply #'insert strings)
+    (add-text-properties start (point) props)))
+
+(defun xref--search-property (property &optional backward)
+    "Search the next text range where text property PROPERTY is non-nil.
+Return the value of PROPERTY.  If BACKWARD is non-nil, search
+backward."
+  (let ((next (if backward
+                  #'previous-single-char-property-change
+                #'next-single-char-property-change))
+        (start (point))
+        (value nil))
+    (while (progn
+             (goto-char (funcall next (point) property))
+             (not (or (setq value (get-text-property (point) property))
+                      (eobp)
+                      (bobp)))))
+    (cond (value)
+          (t (goto-char start) nil))))
+
+\f
+;;; Marker stack  (M-. pushes, M-, pops)
+
+(defconst xref--marker-ring-length 16)
+
+(defvar xref--marker-ring (make-ring xref--marker-ring-length)
+  "Ring of markers to implement the marker stack.")
+
+(defun xref-push-marker-stack ()
+  "Add point to the marker stack."
+  (ring-insert xref--marker-ring (point-marker)))
+
+;;;###autoload
+(defun xref-pop-marker-stack ()
+  "Pop back to where \\[xref-find-definitions] was last invoked."
+  (interactive)
+  (let ((ring xref--marker-ring))
+    (when (ring-empty-p ring)
+      (error "Marker stack is empty"))
+    (let ((marker (ring-remove ring 0)))
+      (switch-to-buffer (or (marker-buffer marker)
+                            (error "The marked buffer has been deleted")))
+      (goto-char (marker-position marker))
+      (set-marker marker nil nil))))
+
+;; etags.el needs this
+(defun xref-clear-marker-stack ()
+  "Discard all markers from the marker stack."
+  (let ((ring xref--marker-ring))
+    (while (not (ring-empty-p ring))
+      (let ((marker (ring-remove ring)))
+        (set-marker marker nil nil)))))
+
+\f
+(defun xref--goto-location (location)
+  "Set buffer and point according to xref-location LOCATION."
+  (let ((marker (xref-location-marker location)))
+    (set-buffer (marker-buffer marker))
+    (cond ((and (<= (point-min) marker) (<= marker (point-max))))
+          (widen-automatically (widen))
+          (t (error "Location is outside accessible part of buffer")))
+    (goto-char marker)))
+
+(defun xref--pop-to-location (location &optional window)
+  "Goto xref-location LOCATION and display the buffer.
+WINDOW controls how the buffer is displayed:
+  nil      -- switch-to-buffer
+  'window  -- pop-to-buffer (other window)
+  'frame   -- pop-to-buffer (other frame)"
+  (xref--goto-location location)
+  (cl-ecase window
+    ((nil)  (switch-to-buffer (current-buffer)))
+    (window (pop-to-buffer (current-buffer) t))
+    (frame  (let ((pop-up-frames t)) (pop-to-buffer (current-buffer) t)))))
+
+\f
+;;; XREF buffer (part of the UI)
+
+;; The xref buffer is used to display a set of xrefs.
+
+(defun xref--display-position (pos other-window recenter-arg)
+  ;; show the location, but don't hijack focus.
+  (with-selected-window (display-buffer (current-buffer) other-window)
+    (goto-char pos)
+    (recenter recenter-arg)))
+
+(defun xref--show-location (location)
+  (condition-case err
+      (progn
+        (xref--goto-location location)
+        (xref--display-position (point) t 1))
+    (user-error (message (error-message-string err)))))
+
+(defun xref--next-line (backward)
+  (let ((loc (xref--search-property 'xref-location backward)))
+    (when loc
+      (xref--show-location loc))))
+
+(defun xref-next-line ()
+  "Move to the next xref and display its source in the other window."
+  (interactive)
+  (xref--next-line nil))
+
+(defun xref-prev-line ()
+  "Move to the previous xref and display its source in the other window."
+  (interactive)
+  (xref--next-line t))
+
+(defun xref--location-at-point ()
+  (or (get-text-property (point) 'xref-location)
+      (error "No reference at point")))
+
+(defun xref-goto-xref ()
+  "Jump to the xref at point and close the xref buffer."
+  (interactive)
+  (xref--show-location (xref--location-at-point))
+  (quit-window))
+
+(define-derived-mode xref--xref-buffer-mode fundamental-mode "XREF"
+  "Mode for displaying cross-refenences."
+  (setq buffer-read-only t))
+
+(let ((map xref--xref-buffer-mode-map))
+  (define-key map (kbd "q") #'quit-window)
+  (define-key map [remap next-line] #'xref-next-line)
+  (define-key map [remap previous-line] #'xref-prev-line)
+  (define-key map (kbd "RET") #'xref-goto-xref)
+
+  ;; suggested by Johan Claesson "to further reduce finger movement":
+  (define-key map (kbd ".") #'xref-next-line)
+  (define-key map (kbd ",") #'xref-prev-line))
+
+(defun xref--buffer-name () "*xref*")
+
+(defun xref--insert-xrefs (xref-alist)
+  "Insert XREF-ALIST in the current-buffer.
+XREF-ALIST is of the form ((GROUP . (XREF ...)) ...).  Where
+GROUP is a string for decoration purposes and XREF is an
+`xref--xref' object."
+  (cl-loop for ((group . xrefs) . more1) on xref-alist do
+           (xref--insert-propertized '(face bold) group "\n")
+           (cl-loop for (xref . more2) on xrefs do
+                    (insert "  ")
+                    (with-slots (description location) xref
+                      (xref--insert-propertized
+                       (list 'xref-location location
+                             'face 'font-lock-keyword-face)
+                       description))
+                    (when (or more1 more2)
+                      (insert "\n")))))
+
+(defun xref--analyze (xrefs)
+  "Find common filenames in XREFS.
+Return an alist of the form ((FILENAME . (XREF ...)) ...)."
+  (xref--alistify xrefs
+                  (lambda (x)
+                    (xref-location-group (xref--xref-location x)))
+                  #'equal))
+
+(defun xref--show-xref-buffer (xrefs)
+  (let ((xref-alist (xref--analyze xrefs)))
+    (with-current-buffer (get-buffer-create (xref--buffer-name))
+      (let ((inhibit-read-only t))
+        (erase-buffer)
+        (xref--insert-xrefs xref-alist)
+        (xref--xref-buffer-mode)
+        (pop-to-buffer (current-buffer))
+        (goto-char (point-min))
+        (current-buffer)))))
+
+\f
+;; This part of the UI seems fairly uncontroversial: it reads the
+;; identifier and deals with the single definition case.
+;;
+;; The controversial multiple definitions case is handed off to
+;; xref-show-xrefs-function.
+
+(defvar xref-show-xrefs-function 'xref--show-xref-buffer
+  "Function to display a list of xrefs.")
+
+(defun xref--show-xrefs (id kind xrefs window)
+  (cond
+   ((null xrefs)
+    (if (eq id t)
+        (error "No known %s for the identifier at point" kind)
+      (error "No known %s for: %s" kind id)))
+   ((not (cdr xrefs))
+    (xref-push-marker-stack)
+    (xref--pop-to-location (xref--xref-location (car xrefs)) window))
+   (t
+    (xref-push-marker-stack)
+    (funcall xref-show-xrefs-function xrefs))))
+
+(defun xref--read-identifier (prompt)
+  "Return the identifier at point or read it from the minibuffer."
+  (let ((id (funcall xref-identifier-at-point-function)))
+    (cond ((or current-prefix-arg (not id))
+           (completing-read prompt (funcall xref-identifier-table-function)
+                            nil t (unless (eq id t) id)))
+          (t id))))
+
+\f
+;;; Commands
+
+(defun xref--find-definitions (id window)
+  (xref--show-xrefs id "definitions"
+                    (funcall xref-find-function 'definitions id)
+                    window))
+
+;;;###autoload
+(defun xref-find-definitions (identifier)
+  "Find the definition of the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier nil))
+
+;;;###autoload
+(defun xref-find-definitions-other-window (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'window))
+
+;;;###autoload
+(defun xref-find-definitions-other-frame (identifier)
+  "Like `xref-find-definitions' but switch to the other window."
+  (interactive (list (xref--read-identifier "Find definitions of: ")))
+  (xref--find-definitions identifier 'frame))
+
+;;;###autoload
+(defun xref-find-references (identifier)
+  "Find references to the identifier at point.
+With prefix argument, prompt for the identifier."
+  (interactive (list (xref--read-identifier "Find references of: ")))
+  (xref--show-xrefs identifier "references"
+                    (funcall xref-find-function 'references identifier)
+                    nil))
+
+;;;###autoload
+(defun xref-find-apropos (pattern)
+  "Find all meaningful symbols that match PATTERN.
+The argument has the same meaning as in `apropos'."
+  (interactive (list (read-from-minibuffer
+                      "Search for pattern (word list or regexp): ")))
+  (require 'apropos)
+  (xref--show-xrefs pattern "apropos"
+                    (funcall xref-find-function 'apropos
+                             (apropos-parse-pattern
+                              (if (string-equal (regexp-quote pattern) pattern)
+                                  ;; Split into words
+                                  (or (split-string pattern "[ \t]+" t)
+                                      (user-error "No word list given"))
+                                pattern)))
+                    nil))
+
+\f
+;;; Key bindings
+
+;;;###autoload
+(progn
+  (define-key esc-map "." #'xref-find-definitions)
+  (define-key esc-map "," #'xref-pop-marker-stack)
+  (define-key ctl-x-4-map "." #'xref-find-definitions-other-window)
+  (define-key ctl-x-5-map "." #'xref-find-definitions-other-frame))
+
+\f
+(provide 'xref)
+
+;;; xref.el ends here

Re: Generalizing find-definition

[Dmitry Gutov]

On 12/11/2014 05:07 PM, Stefan Monnier wrote:

>     (add-function :around (local xref-backend) #'elisp-xref-backend)

Okay, thanks for the reminder about nadvice.

I guess with this approach to function inheritance, there would be a 
different, bare-bones default value of xref-find-function, because I 
don't think I want to delegate to etags-xref-find from elisp-xref-find 
(and all other backends).

Anyway, having a separate xref-identifier-at-point-function should solve 
this problem in a different way.

> This should be called `xref-backend-function', to follow usual practice.

Right, thanks.

> Rename xref--xref to xref-xref, since it's clearly part of the API.

Or we can keep the class name private, and document that the function 
should return a list of xref values, using `xref-make'.

> A few comments:
> - As discussed earlier, `read-identifier' is probably a bad idea.
>    Replace it with a `completion-table' method, so the completing-read is
>    not in the backend but in xref.el.

Ok, apparently I missed some things in the older thread.

> - identifier-at-point should document more clearly that the return value
>    may be something else than a string.  AFAIK the only interesting
>    non-string non-nil case is to return a special value (e.g. a marker
>    or just `t' to stand to (point)) which stands for "let the
>    find-definitions code figure out the identifier at that place".
> - If we restrict identifier to "a string or nil or t", then we can get
>    rid of identifier-to-string.

All right, let's go with this, until a better idea comes up.

But note that AFAIK 99% of the time the identifier at point and 
prefix+suffix used by completion-at-point will be one and the same. 
Maybe it will make sense to unify these: for example, by naming the 
function identifier-bounds, and requiring it to return the beginning and 
the end of the identifier.

This pair of numbers (or maybe markers) would be useful to both 
find-definition functions that expect a string (just call 
buffer-substring beforehand), as well as those that need a buffer position.

>    a single backend function, but adding the other methods makes it
>    scream "I want to use OO", just like completion-tables do.

I wasn't aware they're considered a bad design already. :) That API is 
complicated, but not just because a function has many calling conventions.

>    then we need to split `xref-backend' into several variables:
>    - xref-find-function (can be used both for the `definitions' and for
>      the `references').
>    - xref-identifier-at-point-function.
>    - xref-identifier-completion-table.

Ok. (I should send the updated patch tonight).

Re: Generalizing find-definition

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Helmut Eller <eller.helmut@gmail.com> writes:
>
>> On Mon, Dec 08 2014, Stefan Monnier wrote:
>>
>>> Alright, let's go ahead with this.  Can you send a "latest and greatest"
>>> version of your code so I can install it into master?
>>
>> Here we go:
>
> Would anyone mind if I try my hand at porting this code to the "plain
> functions" approach?

I would; the use of EIEIO was the primary attraction of this version for
me.

> The functionality is already nice, but the standard Emacs APIs probably
> shouldn't force users to get familiar with EIEIO 

This package does not force anyone to know EIEIO; the package API is all
plain functions.

But people can use the classes if they want to; I intend to in ada-mode.

-- 
-- Stephe

Re: Generalizing find-definition

[Dmitry Gutov]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> This package does not force anyone to know EIEIO; the package API is all
> plain functions.

The API mandates that `xref-backend-function' returns an instance of a
class derived from xref-backend-class.  How would you implement the
find-func backend without changing xref?

> But people can use the classes if they want to; I intend to in ada-mode.

I don't see why this can't happen with a plain-function implementation.

Re: Generalizing find-definition

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> This package does not force anyone to know EIEIO; the package API is all
>> plain functions.
>
> The API mandates that `xref-backend-function' returns an instance of a
> class derived from xref-backend-class.  

Ah, right; I forgot about that.

On the other hand, the required code is three lines:

(defclass xref-ada-backend (xref-backend-class) ())
(defvar xref--ada-backend (make-instance 'xref-ada-backend))
(defun xref-ada-backend-function () xref--ada-backend)

xref could provide a macro that does this.

Everything else in a major-mode backend can be non-EIEIO.

>> But people can use the classes if they want to; I intend to in ada-mode.
>
> I don't see why this can't happen with a plain-function implementation.

Perhaps I've misunderstood what you are proposing.

I have the impression you are proposing to entirely delete EIEIO from xref.

What are you proposing?

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> On the other hand, the required code is three lines:

> (defclass xref-ada-backend (xref-backend-class) ())
> (defvar xref--ada-backend (make-instance 'xref-ada-backend))
> (defun xref-ada-backend-function () xref--ada-backend)

You also need to define the corresponding methods.  IOW, all the visible
part of the API relies on EIEIO.


        Stefan

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> On the other hand, the required code is three lines:
>
>> (defclass xref-ada-backend (xref-backend-class) ())
>> (defvar xref--ada-backend (make-instance 'xref-ada-backend))
>> (defun xref-ada-backend-function () xref--ada-backend)
>
> You also need to define the corresponding methods.  IOW, all the visible
> part of the API relies on EIEIO.

Yes.

Sigh, I seem to be particularly dense about this.

-- 
-- Stephe

Re: Generalizing find-definition

[Stefan Monnier]

> Would anyone mind if I try my hand at porting this code to the "plain
> functions" approach?

Go ahead, but try to minimize the changes w.r.t xref.el so it's easier
to compare the two.


        Stefan

Re: Generalizing find-definition

[Dmitry Gutov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Go ahead, but try to minimize the changes w.r.t xref.el so it's easier
> to compare the two.

Sure. I actually changed less than declared: the main change is,
elisp-xref-backend lives in elisp-mode.el. And it's a plain function, of
course.

Re: Generalizing find-definition

[Stephen Leake]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> This said, I think the most urgent step is to get such a generalized
>>> find-definition into Emacs.
>> Here is a patch:
>
> What happened to the other contenders?

I voted for this one; I have seen no other traffic on this.


-- 
-- Stephe

Re: Generalizing find-definition

[Stephen Leake]

Jorgen Schaefer <forcer@forcix.cx> writes:

> So the new module find-definition.el would define:

looks good; ada-mode could use this

ada-mode also has "show all references of name at point", which
currently pops up a buffer in compilation-mode, showing
compilation-style locations of all uses. That's similar functionality to
find-tag/tags-loop-continue, and similar UI to SLIME.

> Changes to elisp-mode.el:
>
> Provide a value for `find-definition-function' which finds the
> definition of the symbol at point.

+1

I have my own hack for this; It requires me to distinguish between
variable and defun names and invoke separate keys; I hope you can remove
that annoyance.

-- 
-- Stephe

Re: Generalizing find-definition

[Jorgen Schaefer]

On Sun, 02 Nov 2014 17:26:08 -0500
Stephen Leake <stephen_leake@stephe-leake.org> wrote:

> Jorgen Schaefer <forcer@forcix.cx> writes:
> 
> > Changes to elisp-mode.el:
> >
> > Provide a value for `find-definition-function' which finds the
> > definition of the symbol at point.
> 
> +1
> 
> I have my own hack for this; It requires me to distinguish between
> variable and defun names and invoke separate keys; I hope you can
> remove that annoyance.

I'm not sure how I can remove that annoyance - is that not
language-dependent? What kind of functionality would you require there?

Regards,
Jorgen

Re: Generalizing find-definition

[Helmut Eller]

On Mon, Nov 03 2014, Jorgen Schaefer wrote:

>> I have my own hack for this; It requires me to distinguish between
>> variable and defun names and invoke separate keys; I hope you can
>> remove that annoyance.
>
> I'm not sure how I can remove that annoyance - is that not
> language-dependent?  What kind of functionality would you require there? 

One way to solve this is to merge the list of candidates for functions
and variables.  At least that's what we do in SLIME.  In elisp-mode
M-. could combine the candidates of find-function and find-variable.
Most of the time the merged list would only contain a single candidate.

Helmut

Re: Generalizing find-definition

[Stephen Leake]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, Nov 03 2014, Jorgen Schaefer wrote:
>
>>> I have my own hack for this; It requires me to distinguish between
>>> variable and defun names and invoke separate keys; I hope you can
>>> remove that annoyance.
>>
>> I'm not sure how I can remove that annoyance - is that not
>> language-dependent?  What kind of functionality would you require there? 
>
> One way to solve this is to merge the list of candidates for functions
> and variables.  At least that's what we do in SLIME.  In elisp-mode
> M-. could combine the candidates of find-function and find-variable.
> Most of the time the merged list would only contain a single candidate.

+1

-- 
-- Stephe

Re: Generalizing find-definition

[Jorgen Schaefer]

On Mon, 03 Nov 2014 09:13:45 +0100
Helmut Eller <eller.helmut@gmail.com> wrote:

> On Mon, Nov 03 2014, Jorgen Schaefer wrote:
> 
> >> I have my own hack for this; It requires me to distinguish between
> >> variable and defun names and invoke separate keys; I hope you can
> >> remove that annoyance.
> >
> > I'm not sure how I can remove that annoyance - is that not
> > language-dependent?  What kind of functionality would you require
> > there? 
> 
> One way to solve this is to merge the list of candidates for functions
> and variables.  At least that's what we do in SLIME.  In elisp-mode
> M-. could combine the candidates of find-function and find-variable.
> Most of the time the merged list would only contain a single
> candidate.

Why would the proposed find-definition interface make a distinction
between variables and functions at all? So far, there has been none
proposed?

Does that mean that this is not an issue? That'd be great :-)

Regards,
Jorgen

Re: Generalizing find-definition

[Stephen Leake]

Jorgen Schaefer <forcer@forcix.cx> writes:

> On Mon, 03 Nov 2014 09:13:45 +0100
> Helmut Eller <eller.helmut@gmail.com> wrote:
>
>> On Mon, Nov 03 2014, Jorgen Schaefer wrote:
>> 
>> >> I have my own hack for this; It requires me to distinguish between
>> >> variable and defun names and invoke separate keys; I hope you can
>> >> remove that annoyance.
>> >
>> > I'm not sure how I can remove that annoyance - is that not
>> > language-dependent?  What kind of functionality would you require
>> > there? 
>> 
>> One way to solve this is to merge the list of candidates for functions
>> and variables.  At least that's what we do in SLIME.  In elisp-mode
>> M-. could combine the candidates of find-function and find-variable.
>> Most of the time the merged list would only contain a single
>> candidate.
>
> Why would the proposed find-definition interface make a distinction
> between variables and functions at all? So far, there has been none
> proposed?

I agree, it should not.

> Does that mean that this is not an issue? 

No, the problem is the backend functions currently provided by elisp;
function-called-at-point vs variable-at-point. There is no current
elisp function that provides 'identifier-at-point'.

In addition, symbol-file requires 'defun or 'defvar to return the
appropriate file name.

The proposed solution for the elisp backend for
find-definition-function is to call both function-called-at-point and
variable-at-point; if only one gives a result, go there. Otherwise, show
the list of two locations.

That's still not quite right; if point is on an argument to a function,
function-called-at-point will return the function symbol, not the
argument. So we really need just identifier-at-point, which is the first
portion of variable-at-point and function-called-at-point.

-- 
-- Stephe