bug#19468: 25.0.50; UI inconveniences with M-.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

Make sure you generated src/TAGS, and then:

  emacs -Q
  C-x C-f src/buffer.c RET
  M-x visit-tags-table RET RET
  M-. display_line RET

This pops up a *xref* buffer which looks like this:

  d:/gnu/git/emacs/trunk/src/xdisp.c
    display_line 
    move_it_in_display_line_to 
    move_it_in_display_line

with point at the beginning of the first line.

Why doesn't it put me on "display_line"s line, and display its
definition at the same time?  This is a regression from the old tags
feature, where just "M-. RET" will already show the first matching
definition.

Further, this buffer's name, *xref*, has no mnemonic significance, and
there are no clues as to what it wants to tell me or what is expected
of me.  The candidates are not mouse-sensitive, either, which is
un-Emacsy.  It functions like *Completions*, but it ain't one.

By trial and error I found out that I'm expected to move to the
candidate I want with cursor movement keys, and press RET to exit the
buffer.  Moving up an down is slow, probably because it visits files
without waiting for RET or some other gesture to select a candidate
(why was that design decision made?).  Hitting RET on the first line,
the one that shows the file name, results in "No reference at point",
which is not really useful.

Another peculiarity is that once I press <down> arrow once, I can no
longer get back to the first line, the one that shows the source file:
pressing <up> on the 2nd line doesn't move point, but it does return
the original buffer to the window above *xref*.  Weird.

Finally, invoking the same command from the menu bar ought to present
a dialog box with the candidates, according to the general rule: if a
command was invoked by a mouse gesture, selection of candidates is via
a GUI dialog, not a special-purpose buffer.  But that doesn't happen.

In sum: please make the new feature at least as good as the old one it
replaces.  And when introducing new exhibits, like the *xref* window,
please make them self-explanatory and convenient/natural to use for
newbies and veterans alike.

TIA


In GNU Emacs 25.0.50.110 (i686-pc-mingw32)
 of 2014-12-29 on HOME-C4E4A596F7
Repository revision: ce1ebdf1ba8acc75e8f959f414652cdc87e76401
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --prefix=/d/usr --enable-checking=yes,glyphs 'CFLAGS=-O0
 -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: XREF

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Starting a new list of tags tables
M-. runs the command xref-find-definitions
M-, runs the command xref-pop-marker-stack
Making tags completion table for d:/gnu/git/emacs/trunk/src/TAGS...80%
Making tags completion table for d:/gnu/git/emacs/trunk/lisp/TAGS...94%
Making tags completion table for d:/gnu/git/emacs/trunk/lwlib/TAGS...
Making tags completion table for d:/gnu/git/emacs/trunk/src/TAGS...done
Making completion list...
xref--location-at-point: No reference at point

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message dired format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils cl-macs gv thingatpt etags xref
eieio byte-opt bytecomp byte-compile cl-extra cconv eieio-core
cl-loaddefs cl-lib ring vc-git diff-mode easy-mmode cc-mode cc-fonts
easymenu cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs time-date tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table w32-win w32-vars
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer nadvice loaddefs button faces cus-face macroexp
files text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)

Memory information:
((conses 8 123285 11708)
 (symbols 32 104663 1)
 (miscs 32 77 302)
 (strings 16 108087 9345)
 (string-bytes 1 2510300)
 (vectors 8 15137)
 (vector-slots 4 446128 6934)
 (floats 8 75 209)
 (intervals 28 2628 236)
 (buffers 516 18))

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/29/2014 10:26 PM, Eli Zaretskii wrote:

> Why doesn't it put me on "display_line"s line, and display its
> definition at the same time?  This is a regression from the old tags
> feature, where just "M-. RET" will already show the first matching
> definition.

It's a tradeoff. Otherwise, you'd see two new buffers at once, possibly 
covering both windows (if you have two). And it seems inconsistent with 
your further complaint that movement visits files automatically (if it 
shouldn't, it shouldn't start with displaying the buffer at point either).

Compared to `find-tag', you see the list of all possible definitions 
(and only if there are several). What are the odds that the first 
location is the right one anyway?

> Further, this buffer's name, *xref*, has no mnemonic significance, and
> there are no clues as to what it wants to tell me or what is expected
> of me.

Would you like it to be called the same as the current command? 
*xref-find-definitions-other-window*, *xref-find-apropos*, etc?

> The candidates are not mouse-sensitive, either, which is
> un-Emacsy.  It functions like *Completions*, but it ain't one.

One might argue that using the mouse is also un-Emacsy. But sure, that 
shouldn't be hard to add (and would increase discoverability).

> By trial and error I found out that I'm expected to move to the
> candidate I want with cursor movement keys,

Or with `.' and `,', which are a bit easier to press right after `M-.'.

 > Moving up an down is slow, probably because it visits files
> without waiting for RET or some other gesture to select a candidate
> (why was that design decision made?).

If you look at the related thread in emacs-devel, you'll see that it 
hasn't been discussed at all. Helmut implemented it this way, and 
apparently everyone that looked at it (the few people that did), liked 
it. I expect it will have its admirers, since helm-swoop, a package with 
the same visual effect, is pretty popular.

This can be made configurable, though. For instance, `C-o' could be that 
"other gesture". Would you prefer the window configuration restored 
before point moves to a different line, or should the new buffer keep 
being displayed? The latter presents a challenge if we want `q' in the 
xref buffer to restore the original window configuration before 
xref-find-definitions was invoked, as long as the user hadn't made any 
manual changes thereafter.

 > Hitting RET on the first line,
> the one that shows the file name, results in "No reference at point",
> which is not really useful.

Would you prefer to navigate to the first line of that file? That seems 
unlikely.

> Another peculiarity is that once I press <down> arrow once, I can no
> longer get back to the first line, the one that shows the source file:
> pressing <up> on the 2nd line doesn't move point, but it does return
> the original buffer to the window above *xref*.  Weird.

A bit weird, yes. Would you prefer not to "return the original buffer"?

> Finally, invoking the same command from the menu bar ought to present
> a dialog box with the candidates, according to the general rule: if a
> command was invoked by a mouse gesture, selection of candidates is via
> a GUI dialog, not a special-purpose buffer.  But that doesn't happen.

Example?

Since when do we have a working completion interface that uses a GUI 
dialog ("open file" doesn't count)? If it's as fully functional, then 
sure, we should use that.

As a counterpoint, Buffers->Select Named Buffer... uses the minibuffer.

> In sum: please make the new feature at least as good as the old one it
> replaces.

I'd say it's already much better, but maybe not in all respects. And the 
latter would be a tough goal.

> And when introducing new exhibits, like the *xref* window,
> please make them self-explanatory and convenient/natural to use for
> newbies and veterans alike.

That's a great ideal to strive for, but a poor criterion for acceptance 
(convenient and natural are subjective notions). When the veterans don't 
participate in the discussion about a new feature, I suspect having 
follow-up discussions in bug reports would often be inevitable.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Dmitry Gutov wrote:

> On 12/29/2014 10:26 PM, Eli Zaretskii wrote:
>
>> Why doesn't it put me on "display_line"s line, and display its
>> definition at the same time?

As Dmitry says: that would replace the current buffer and at the same
time create and switch to the *xref* buffer.  I had tried that and it
didn't like it.

>> This is a regression from the old tags
>> feature, where just "M-. RET" will already show the first matching
>> definition.

It's not a regression as M-. is now a different command with a different
UI.  My goal was never to be 100% backward compatible with find-tag but
to make something better.  You lost your key binding for find-tag.  Sorry
about that, but there are only so many good keys.  Of course, you know
how to get the old key binding back.

> It's a tradeoff. Otherwise, you'd see two new buffers at once,
> possibly covering both windows (if you have two). And it seems
> inconsistent with your further complaint that movement visits files
> automatically (if it shouldn't, it shouldn't start with displaying the
> buffer at point either).
>
> Compared to `find-tag', you see the list of all possible definitions
> (and only if there are several). What are the odds that the first
> location is the right one anyway?
>
>> Further, this buffer's name, *xref*, has no mnemonic significance, and
>> there are no clues as to what it wants to tell me or what is expected
>> of me.
>
> Would you like it to be called the same as the current command?
> *xref-find-definitions-other-window*, *xref-find-apropos*, etc?

I bet that after 15 minutes of using it, nobody will care what the name
of the *xref* buffer is.  So we can just as well use something short.

[...]
>> By trial and error I found out that I'm expected to move to the
>> candidate I want with cursor movement keys,

Trial and error isn't the worst way to learn things; at least if it
doesn't take too long.  Apparently you didn't even need to read any
documentation, which would indicate that the UI is not so unintuitive
after all.

[...]
>> Moving up an down is slow, probably because it visits files
>> without waiting for RET or some other gesture to select a candidate

For me, opening the file the first time is a bit slow, but after that
moving up and down is instantaneous.

[...]
>> Another peculiarity is that once I press <down> arrow once, I can no
>> longer get back to the first line, the one that shows the source
>> file:

That's as expected.  There is nothing to select on the first line.

>> pressing <up> on the 2nd line doesn't move point, but it does return
>> the original buffer to the window above *xref*.  Weird.
>
> A bit weird, yes. Would you prefer not to "return the original buffer"?

I didn't implement that.  It's because of the save-window-excursion in
xref--next-line that Dmitry added.  In my proposal the other window
didn't change when the cursor didn't move.

>> In sum: please make the new feature at least as good as the old one it
>> replaces.

You can't make progress by keeping everything the same.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 30 Dec 2014 08:04:41 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: Helmut Eller <eller.helmut@gmail.com>
> 
> On 12/29/2014 10:26 PM, Eli Zaretskii wrote:
> 
>     Why doesn't it put me on "display_line"s line, and display its
>     definition at the same time?  This is a regression from the old tags
>     feature, where just "M-. RET" will already show the first matching
>     definition.
> 
> 
> It's a tradeoff. Otherwise, you'd see two new buffers at once, possibly covering both windows (if you have two).

Sorry, I don't follow.  I started that from a single window, and
M-. already did split it into 2, and displayed the candidates in the
lower one.  So I already see 2 buffers, and showing the first
candidate in the upper window doesn't do anything I won't expect --
after all, I did ask to see something different from what I was
already seeing.

> And it seems inconsistent with your further complaint that movement visits files automatically (if it shouldn't, it shouldn't start with displaying the buffer at point either).

I see no inconsistency: we could display the first candidate
automatically, but not switch to others without the user's say-so.

Imagine a situation where the list of possible candidates is very
long, longer than what the window shows.  Then the user might wish to
scroll the list of candidates before she makes up her mind about the
one she wants to see.  But moving point visits the candidates, which
is not what the user asked for in this use case.

Moreover, I see that scrolling the window by C-v/PageDn does _not_
visit the tags at point, while scrolling by up- and down- arrow key
does.  Looks like another confusing inconsistency.

So I suggest that the automatic visiting will be at least
customizable, if not switched off by default.

> Compared to `find-tag', you see the list of all possible definitions (and only if there are several). What are the odds that the first location is the right one anyway?

With completion? excellent.  Moreover, the old find-tags employed some
logic in its completion, so that matches that are more likely appear
first.  I generally found the results very good, certainly in
languages which require symbols to have unique names on the source
level.

>     Further, this buffer's name, *xref*, has no mnemonic significance, and
>     there are no clues as to what it wants to tell me or what is expected
>     of me.
> 
> 
> Would you like it to be called the same as the current command? *xref-find-definitions-other-window*, *xref-find-apropos*, etc?

How about "*find-function-candidates*" or even "*Completions*"?

>     The candidates are not mouse-sensitive, either, which is
>     un-Emacsy.  It functions like *Completions*, but it ain't one.
> 
> 
> One might argue that using the mouse is also un-Emacsy.

That ship sailed a long time ago.

> But sure, that shouldn't be hard to add (and would increase discoverability).

More importantly, it will tell the user what she needs to do, even if
she doesn't use the mouse, because mouse-sensitive portions of Emacs
display generally behave very similarly.

>     By trial and error I found out that I'm expected to move to the
>     candidate I want with cursor movement keys,
> 
> 
> Or with `.' and `,', which are a bit easier to press right after `M-.'.

It would be nice to have some instructions to this effect.

Once again, this is not a UI that is used in other places in Emacs, it
is very different.  So you shouldn't assume that its keybindings are
known by users, especially keybindings such as '.' and ',' that are
not widely used in other programs, and therefore must be learned anew.

> > Moving up an down is slow, probably because it visits files
> 
>     without waiting for RET or some other gesture to select a candidate
>     (why was that design decision made?).
> 
> If you look at the related thread in emacs-devel, you'll see that it hasn't been discussed at all. Helmut implemented it this way, and apparently everyone that looked at it (the few people that did), liked it. I expect it will have its admirers, since helm-swoop, a package with the same visual effect, is pretty popular.

I described above one use case where I think this reaction to moving
point will not be appreciated by users, since Emacs moves point as
result of scrolling.

> This can be made configurable, though. For instance, `C-o' could be that "other gesture". Would you prefer the window configuration restored before point moves to a different line, or should the new buffer keep being displayed? The latter presents a challenge if we want `q' in the xref buffer to restore the original window configuration before xref-find-definitions was invoked, as long as the user hadn't made any manual changes thereafter.

Why is it a challenge?  Typing 'q' and 'C-o' can invoke different
functions, can't it?  I'm probably missing something.

> > Hitting RET on the first line,
> 
>     the one that shows the file name, results in "No reference at point",
>     which is not really useful.
> 
> 
> Would you prefer to navigate to the first line of that file? That seems unlikely.

My point is that putting me on that line when the buffer pops up makes
little sense, because that line is just a heading.

>     Another peculiarity is that once I press <down> arrow once, I can no
>     longer get back to the first line, the one that shows the source file:
>     pressing <up> on the 2nd line doesn't move point, but it does return
>     the original buffer to the window above *xref*.  Weird.
> 
> A bit weird, yes. Would you prefer not to "return the original buffer"?

No, I prefer that the lines that show file names be not reachable at
all.  They are just clutter, as far as moving point is concerned.  Let
point jump over them.

>     Finally, invoking the same command from the menu bar ought to present
>     a dialog box with the candidates, according to the general rule: if a
>     command was invoked by a mouse gesture, selection of candidates is via
>     a GUI dialog, not a special-purpose buffer.  But that doesn't happen.
> 
> Example?

Click File->Quit when you have unsaved edits.

> Since when do we have a working completion interface that uses a GUI dialog ("open file" doesn't count)? If it's as fully functional, then sure, we should use that.

But this is not completion, this is a list of completion results.
Completion happens _before_ the *xref* buffer is popped up.  Or am I
missing something?

> As a counterpoint, Buffers->Select Named Buffer... uses the minibuffer.

That _is_ completion.  I'm not talking about prompting the user for a
symbol, I'm talking about _displaying_ the matching symbols once the
user has typed her input.

>     In sum: please make the new feature at least as good as the old one it
>     replaces.
> 
> I'd say it's already much better, but maybe not in all respects. And the latter would be a tough goal.

I don't think we can deprecate/obsolete find-tag unless xref is at
least as good.  And I don't see any reasons why it couldn't become as
good or better, the issues I mentioned seem to be minor and easy to
fix.

>     And when introducing new exhibits, like the *xref* window,
>     please make them self-explanatory and convenient/natural to use for
>     newbies and veterans alike.
> 
> That's a great ideal to strive for, but a poor criterion for acceptance (convenient and natural are subjective notions). 

I agree it's subjective, but I think I provided explanations for my
views.

> When the veterans don't participate in the discussion about a new feature, I suspect having follow-up discussions in bug reports would often be inevitable.

I'm sorry I didn't participate.  The reason is that I never understood
this is meant as a replacement for find-tag etc., until you asked
whether to remove them from the menu bar.  The discussion was long
enough and focused on technicalities enough to turn me off.  Perhaps a
prominent announcement of the intent to replace Tags, with explicit
reference to that word in the Subject, would have done a better job of
attracting people who might have had something to say about this.

Thanks.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: Dmitry Gutov <dgutov@yandex.ru>, 19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 09:19:18 +0100
> 
> >> Why doesn't it put me on "display_line"s line, and display its
> >> definition at the same time?
> 
> As Dmitry says: that would replace the current buffer and at the same
> time create and switch to the *xref* buffer.  I had tried that and it
> didn't like it.

But that's what the old M-. did, so it should be at least an option,
IMO.

> >> This is a regression from the old tags
> >> feature, where just "M-. RET" will already show the first matching
> >> definition.
> 
> It's not a regression as M-. is now a different command with a different
> UI.  My goal was never to be 100% backward compatible with find-tag but
> to make something better.  You lost your key binding for find-tag.  Sorry
> about that, but there are only so many good keys.  Of course, you know
> how to get the old key binding back.

But etc/NEWS says that find-tag is now obsolete.  So I think the
replacement should be at least as good, in terms of usability and
speed.  I didn't ask for 100% backward compatibility, I'm asking for
convenience and speed.  I don't think they are unreasonable requests.

> >> Further, this buffer's name, *xref*, has no mnemonic significance, and
> >> there are no clues as to what it wants to tell me or what is expected
> >> of me.
> >
> > Would you like it to be called the same as the current command?
> > *xref-find-definitions-other-window*, *xref-find-apropos*, etc?

> I bet that after 15 minutes of using it, nobody will care what the name
> of the *xref* buffer is.  So we can just as well use something short.

I don't see how the length of the buffer's name is important here.
Short names are okay if they explain themselves; this one doesn't.

We periodically have discussions full of flames about discoverability
in Emacs and its steep learning curve.  It is my impression as a user
that the name of this buffer, its specialized keybindings, and lack of
instructions, don't make the curve less steep, to say the least.  I
hope we can do better, as the problems don't seem to be grave or hard
to fix.

> >> By trial and error I found out that I'm expected to move to the
> >> candidate I want with cursor movement keys,
> 
> Trial and error isn't the worst way to learn things; at least if it
> doesn't take too long.

Turns out I missed '.' and ',', though.  So trial and error are
evidently less efficient than we would want.

> Apparently you didn't even need to read any documentation

What documentation?  These commands are not yet in the manual.

Or maybe you meant this:

  M-. runs the command xref-find-definitions (found in global-map),
  which is an interactive autoloaded Lisp function in `xref.el'.

  It is bound to M-., <menu-bar> <edit> <goto> <xref-find-def>.

  (xref-find-definitions IDENTIFIER)

  Find the definition of the identifier at point.
  With prefix argument, prompt for the identifier.

That's the entire doc string of the command, all of it.  (Btw, it
doesn't even say that if no identifier is found at point, it will
prompt even without an argument.)

I hope now you understand why I needed to "try and err".

> >> Another peculiarity is that once I press <down> arrow once, I can no
> >> longer get back to the first line, the one that shows the source
> >> file:
> 
> That's as expected.  There is nothing to select on the first line.

Then why let me position point there?

> >> In sum: please make the new feature at least as good as the old one it
> >> replaces.
> 
> You can't make progress by keeping everything the same.

Once again, no one asked for "the same".  I'm asking for the new
feature to be at least as convenient and fast as the one it replaces.

The two issues that I find annoying about this are: the first
candidate is not displayed until I press a key, and I'm unable to find
functions that the current major mode doesn't know about.  I hope
these can be fixed, they don't seem major to me.

Thanks.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Eli Zaretskii wrote:

>> From: Helmut Eller <eller.helmut@gmail.com>
>> Cc: Dmitry Gutov <dgutov@yandex.ru>, 19468@debbugs.gnu.org
>> Date: Tue, 30 Dec 2014 09:19:18 +0100
>> 
>> >> Why doesn't it put me on "display_line"s line, and display its
>> >> definition at the same time?
>> 
>> As Dmitry says: that would replace the current buffer and at the same
>> time create and switch to the *xref* buffer.  I had tried that and it
>> didn't like it.
>
> But that's what the old M-. did,

No, find-tag didn't display a list of candidates.

> so it should be at least an option,
> IMO.

As I said: been there, done that, didn't like it.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

> It's a tradeoff. Otherwise, you'd see two new buffers at once, possibly
> covering both windows (if you have two). And it seems inconsistent with your
> further complaint that movement visits files automatically (if it shouldn't,
> it shouldn't start with displaying the buffer at point either).

I thought the problem that Eli was pointing out is that he requested
"display_line" and there is only one "display_line", so instead of
showing 3 entries (each one including "display_line" as a substring) we
should directly jump to the perfect match.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 07:03 PM, Stefan Monnier wrote:

> I thought the problem that Eli was pointing out is that he requested
> "display_line" and there is only one "display_line", so instead of
> showing 3 entries (each one including "display_line" as a substring) we
> should directly jump to the perfect match.

Not really (see his response), but indeed, filtering out imperfect 
matches retrieved from tags file would be an improvement.

Suppose there are several "perfect" matches. Should the imperfect 
matches be included in the list too, then?

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Stefan Monnier wrote:

>> It's a tradeoff. Otherwise, you'd see two new buffers at once, possibly
>> covering both windows (if you have two). And it seems inconsistent with your
>> further complaint that movement visits files automatically (if it shouldn't,
>> it shouldn't start with displaying the buffer at point either).
>
> I thought the problem that Eli was pointing out is that he requested
> "display_line" and there is only one "display_line", so instead of
> showing 3 entries (each one including "display_line" as a substring) we
> should directly jump to the perfect match.

That's not an UI issue, that is a backend issue.  The etags backend
produces a high number of false positives.  That's not new and as in the
past it can be fixed with a more strict find-tag-tag-order.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: dgutov@yandex.ru,  19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 18:02:59 +0100
> 
> On Tue, Dec 30 2014, Eli Zaretskii wrote:
> 
> >> From: Helmut Eller <eller.helmut@gmail.com>
> >> Cc: Dmitry Gutov <dgutov@yandex.ru>, 19468@debbugs.gnu.org
> >> Date: Tue, 30 Dec 2014 09:19:18 +0100
> >> 
> >> >> Why doesn't it put me on "display_line"s line, and display its
> >> >> definition at the same time?
> >> 
> >> As Dmitry says: that would replace the current buffer and at the same
> >> time create and switch to the *xref* buffer.  I had tried that and it
> >> didn't like it.
> >
> > But that's what the old M-. did,
> 
> No, find-tag didn't display a list of candidates.

I meant the "display the first candidate" part.

> > so it should be at least an option,
> > IMO.
> 
> As I said: been there, done that, didn't like it.

But that's what options are for: to allow different people customize
Emacs to their taste.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  19468@debbugs.gnu.org,  Helmut Eller <eller.helmut@gmail.com>
> Date: Tue, 30 Dec 2014 12:03:30 -0500
> 
> > It's a tradeoff. Otherwise, you'd see two new buffers at once, possibly
> > covering both windows (if you have two). And it seems inconsistent with your
> > further complaint that movement visits files automatically (if it shouldn't,
> > it shouldn't start with displaying the buffer at point either).
> 
> I thought the problem that Eli was pointing out is that he requested
> "display_line" and there is only one "display_line", so instead of
> showing 3 entries (each one including "display_line" as a substring) we
> should directly jump to the perfect match.

Indeed, that's what I meant.

It is okay to display other, less perfect, matches as well, but the
exact match should be displayed without asking me to type anything
else.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 30 Dec 2014 19:11:24 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: Eli Zaretskii <eliz@gnu.org>, 19468@debbugs.gnu.org,
>  Helmut Eller <eller.helmut@gmail.com>
> 
> Suppose there are several "perfect" matches. Should the imperfect 
> matches be included in the list too, then?

Find-tag, when called with an argument, would go through all the
possible matches one by one, starting from the "perfect" one.  So yes,
I think it is good to present the list of all the candidates, it will
be functionally equivalent to what find-tag did.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 07:11 PM, Dmitry Gutov wrote:
> Should the imperfect  matches be included in the list too, then?

Actually, should the imperfect matches be included in the "find 
definitions" list at all, even if there are no perfect ones?

I'd say no, even though `find-tag' does that, and its users might expect it.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Eli Zaretskii wrote:

>> I thought the problem that Eli was pointing out is that he requested
>> "display_line" and there is only one "display_line", so instead of
>> showing 3 entries (each one including "display_line" as a substring) we
>> should directly jump to the perfect match.
>
> Indeed, that's what I meant.
>
> It is okay to display other, less perfect, matches as well, but the
> exact match should be displayed without asking me to type anything
> else.

In other words: you want the old UI back.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 07:41 PM, Eli Zaretskii wrote:
> It is okay to display other, less perfect, matches as well,

Being magnanimous here doesn't help. It's the xref API: if the backend 
returns one result, we jump to it. Otherwise, show a list.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  dgutov@yandex.ru,  19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 18:50:41 +0100
> 
> On Tue, Dec 30 2014, Eli Zaretskii wrote:
> 
> >> I thought the problem that Eli was pointing out is that he requested
> >> "display_line" and there is only one "display_line", so instead of
> >> showing 3 entries (each one including "display_line" as a substring) we
> >> should directly jump to the perfect match.
> >
> > Indeed, that's what I meant.
> >
> > It is okay to display other, less perfect, matches as well, but the
> > exact match should be displayed without asking me to type anything
> > else.
> 
> In other words: you want the old UI back.

Not the old UI, the old functionality, at least the part that
displayed the first match immediately.  I don't mind the list of the
candidates and the new method of switching to the other candidates,
which are new with this UI.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 30 Dec 2014 19:53:58 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: eller.helmut@gmail.com, 19468@debbugs.gnu.org
> 
> On 12/30/2014 07:41 PM, Eli Zaretskii wrote:
> > It is okay to display other, less perfect, matches as well,
> 
> Being magnanimous here doesn't help. It's the xref API: if the backend 
> returns one result, we jump to it. Otherwise, show a list.

Why cannot the command jump to the first result, after it shows the
list?

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 30 Dec 2014 19:49:16 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: Helmut Eller <eller.helmut@gmail.com>, 19468@debbugs.gnu.org
> 
> On 12/30/2014 07:11 PM, Dmitry Gutov wrote:
> > Should the imperfect  matches be included in the list too, then?
> 
> Actually, should the imperfect matches be included in the "find 
> definitions" list at all, even if there are no perfect ones?
> 
> I'd say no

Why not?

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Eli Zaretskii wrote:

> Why cannot the command jump to the first result, after it shows the
> list?

Because then you have a window that needs to be closed.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 07:58 PM, Eli Zaretskii wrote:

> Not the old UI, the old functionality, at least the part that
> displayed the first match immediately.

The first match is only one `.' keypress away. Surely that's not a major 
issue?

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

> That's not an UI issue, that is a backend issue.

Good,


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  monnier@iro.umontreal.ca,  19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 19:01:55 +0100
> 
> On Tue, Dec 30 2014, Eli Zaretskii wrote:
> 
> > Why cannot the command jump to the first result, after it shows the
> > list?
> 
> Because then you have a window that needs to be closed.

Sorry, I don't understand: what window needs to be closed?

What I meant is to do automatically what moving to the first candidate
does.  It doesn't close any windows, AFAICS, it just displays the
definition in the upper window, leaving the list in the lower one.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Eli Zaretskii wrote:

>> On Tue, Dec 30 2014, Eli Zaretskii wrote:
>> 
>> > Why cannot the command jump to the first result, after it shows the
>> > list?
>> 
>> Because then you have a window that needs to be closed.
>
> Sorry, I don't understand: what window needs to be closed?

The window that displays the *xref* buffer.

> What I meant is to do automatically what moving to the first candidate
> does.  It doesn't close any windows, AFAICS, it just displays the
> definition in the upper window, leaving the list in the lower one.

If you press RET in *xref* buffer it closes the *xref* window.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: dgutov@yandex.ru,  monnier@iro.umontreal.ca,  19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 19:20:09 +0100
> 
> > What I meant is to do automatically what moving to the first candidate
> > does.  It doesn't close any windows, AFAICS, it just displays the
> > definition in the upper window, leaving the list in the lower one.
> 
> If you press RET in *xref* buffer it closes the *xref* window.

I meant to simulate the '.' key, not the RET key.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Eli Zaretskii wrote:

>> > What I meant is to do automatically what moving to the first candidate
>> > does.  It doesn't close any windows, AFAICS, it just displays the
>> > definition in the upper window, leaving the list in the lower one.
>> 
>> If you press RET in *xref* buffer it closes the *xref* window.
>
> I meant to simulate the '.' key, not the RET key.

Then this question was already answered 3 times and here's the fourth
time: it would display two buffers at the same time which is confusing
and I had implemented that already and I didn't like it.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 08:27 PM, Eli Zaretskii wrote:

>> If you press RET in *xref* buffer it closes the *xref* window.
>
> I meant to simulate the '.' key, not the RET key.

You should try this:

(advice-add 'xref--show-xref-buffer :after
             (lambda (&rest r)
               (run-with-idle-timer 0 nil #'xref-next-line)))

Before adding the user option, I'd like to see at least one other person 
say that they'll use it.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: dgutov@yandex.ru,  monnier@iro.umontreal.ca,  19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 19:33:07 +0100
> 
> On Tue, Dec 30 2014, Eli Zaretskii wrote:
> 
> >> > What I meant is to do automatically what moving to the first candidate
> >> > does.  It doesn't close any windows, AFAICS, it just displays the
> >> > definition in the upper window, leaving the list in the lower one.
> >> 
> >> If you press RET in *xref* buffer it closes the *xref* window.
> >
> > I meant to simulate the '.' key, not the RET key.
> 
> Then this question was already answered 3 times and here's the fourth
> time: it would display two buffers at the same time which is confusing
> and I had implemented that already and I didn't like it.

I'm not asking you to like it.  I'm asking to provide a user option to
get this behavior, for those who might like it.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Eli Zaretskii wrote:

> I'm not asking you to like it.  I'm asking to provide a user option to
> get this behavior, for those who might like it.

Yay, more customization variables is what we need.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: dgutov@yandex.ru,  monnier@iro.umontreal.ca,  19468@debbugs.gnu.org
> Date: Tue, 30 Dec 2014 19:53:49 +0100
> 
> On Tue, Dec 30 2014, Eli Zaretskii wrote:
> 
> > I'm not asking you to like it.  I'm asking to provide a user option to
> > get this behavior, for those who might like it.
> 
> Yay, more customization variables is what we need.

You can add the behavior I asked for unconditionally, without adding
an option.  I won't mind.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 10:19 AM, Helmut Eller wrote:

> For me, opening the file the first time is a bit slow, but after that
> moving up and down is instantaneous.

Same here.

> I didn't implement that.  It's because of the save-window-excursion in
> xref--next-line that Dmitry added.  In my proposal the other window
> didn't change when the cursor didn't move.

This is a possibly hacky execution of the following idea: I can move up 
and down the lines and see their buffers displayed, but if I haven't 
pressed RET, I should be able to press `q' and the window configuration 
should return to just the way it was before `xref-find-definitions' was 
called.

But not if the user modifier the window configuration in other ways, 
while *xref* buffer was open, or if the user switches to an existing 
*xref* buffer again after burying it.

Suggestions for a better implementation are welcome. Should we mess 
around with `window-configuration-change-hook'?

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 06:00 PM, Eli Zaretskii wrote:

>> Trial and error isn't the worst way to learn things; at least if it
>> doesn't take too long.
>
> Turns out I missed '.' and ',', though.  So trial and error are
> evidently less efficient than we would want.

I guess this will be in the manual, which I'd be happy not to have to 
write at all, and probably won't write until the 25 feature freeze.

There's probably nothing to discuss about this further.

>> Apparently you didn't even need to read any documentation
>
> What documentation?

Any documentation. :)

> (Btw, it
> doesn't even say that if no identifier is found at point, it will
> prompt even without an argument.)

It does now, thanks.

>> That's as expected.  There is nothing to select on the first line.
>
> Then why let me position point there?

Because then you're ideally positioned to press `.' and see the buffer 
of the first match displayed.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 05:41 PM, Eli Zaretskii wrote:

> Sorry, I don't follow.  I started that from a single window, and
> M-. already did split it into 2, and displayed the candidates in the
> lower one.  So I already see 2 buffers, and showing the first
> candidate in the upper window doesn't do anything I won't expect --
> after all, I did ask to see something different from what I was
> already seeing.

Changing buffer in two windows at once, like Helmut said, is a more 
radical change, more likely to confuse the user.

And you lose more context. In the example of ELisp, you could instead 
glance at the original buffer, see that you've jumped from a function 
call (so you need a function), and then simply choose the `defun' among 
the options, without even looking at its buffer.

>> And it seems inconsistent with your further complaint that movement visits files automatically (if it shouldn't, it shouldn't start with displaying the buffer at point either).
>
> I see no inconsistency: we could display the first candidate
> automatically, but not switch to others without the user's say-so.

Inconsistency would be this: you either automatically display the buffer 
of the candidate at point or not. You're asking to do it differently in 
quite similar contexts.

> Imagine a situation where the list of possible candidates is very
> long, longer than what the window shows.  Then the user might wish to
> scroll the list of candidates before she makes up her mind about the
> one she wants to see.  But moving point visits the candidates, which
> is not what the user asked for in this use case.

I would use C-s for that, or indeed C-v/M-v.

> Moreover, I see that scrolling the window by C-v/PageDn does _not_
> visit the tags at point, while scrolling by up- and down- arrow key
> does.  Looks like another confusing inconsistency.

This would be quite easy to solve, if displaying the tag at point after 
any movement command is indeed what we want to do.

> So I suggest that the automatic visiting will be at least
> customizable, if not switched off by default.

Here's a possible compromise: only do that automatically for `.' and 
`,', as well as provide a manual command (say, bound to C-o).

Speaking of the latter, do you see the value in undoing the window 
configuration change before the next command? We could expand this 
behavior to other `-display-' commands with the same binding.

>> Would you like it to be called the same as the current command? *xref-find-definitions-other-window*, *xref-find-apropos*, etc?
>
> How about "*find-function-candidates*" or even "*Completions*"?

`find-function-candidates' doesn't match the command name, nor would it 
be the same for different commands in the xref package. Inconvenient, 
from the implementation's standpoint.

*Completions* is bad, because we're not completing text input.

>> One might argue that using the mouse is also un-Emacsy.
>
> That ship sailed a long time ago.

Guess so.

>> But sure, that shouldn't be hard to add (and would increase discoverability).
>
> More importantly, it will tell the user what she needs to do, even if
> she doesn't use the mouse, because mouse-sensitive portions of Emacs
> display generally behave very similarly.

If she doesn't use the mouse, how will she know the mouse-sensitive 
portions?

>> Or with `.' and `,', which are a bit easier to press right after `M-.'.
>
> It would be nice to have some instructions to this effect.

I don't think having the instructions inside the buffer would be good. 
Unlike `report-emacs-bug', it can be used very frequently, and seeing 
the same simple instructions you already know is annoying.

> Once again, this is not a UI that is used in other places in Emacs, it
> is very different.  So you shouldn't assume that its keybindings are
> known by users, especially keybindings such as '.' and ',' that are
> not widely used in other programs, and therefore must be learned anew.

Should we expand it to other places in Emacs? Suggestions welcome.

>> This can be made configurable, though. For instance, `C-o' could be that "other gesture". Would you prefer the window configuration restored before point moves to a different line, or should the new buffer keep being displayed? The latter presents a challenge if we want `q' in the xref buffer to restore the original window configuration before xref-find-definitions was invoked, as long as the user hadn't made any manual changes thereafter.
>
> Why is it a challenge?  Typing 'q' and 'C-o' can invoke different
> functions, can't it?  I'm probably missing something.

The challenge is to preserve the window configuration, albeit only when 
it makes sense.

> No, I prefer that the lines that show file names be not reachable at
> all.  They are just clutter, as far as moving point is concerned.  Let
> point jump over them.

If the point is at the first reference already, and we've configured 
xref not to display its buffer automatically, the user will have to 
reach for `C-o' instead of `.', to have it displayed.

>>      Finally, invoking the same command from the menu bar ought to present
>>      a dialog box with the candidates, according to the general rule: if a
>>      command was invoked by a mouse gesture, selection of candidates is via
>>      a GUI dialog, not a special-purpose buffer.  But that doesn't happen.
>>
>> Example?
>
> Click File->Quit when you have unsaved edits.

Apparently, this is easier to implement using the native dialogs. How 
would you create a replacement for the *xref* buffer, without losing the 
grouping or ./, functionality?

> But this is not completion, this is a list of completion results.
> Completion happens _before_ the *xref* buffer is popped up.  Or am I
> missing something?

I've just misunderstood. "Selection of candidates" makes me think of 
completion.

>> As a counterpoint, Buffers->Select Named Buffer... uses the minibuffer.
>
> That _is_ completion.  I'm not talking about prompting the user for a
> symbol, I'm talking about _displaying_ the matching symbols once the
> user has typed her input.

Well, how about Buffers->List All Buffers, then?

> I'm sorry I didn't participate.  The reason is that I never understood
> this is meant as a replacement for find-tag etc., until you asked
> whether to remove them from the menu bar.  The discussion was long
> enough and focused on technicalities enough to turn me off.

The intention to replace find-tag came naturally later. Either way, 
maybe it's good that you didn't get into technicalities then, and 
evaluate this now purely on the basis of behavior.

bug#19468: 25.0.50; UI inconveniences with M-.

[Helmut Eller]

On Tue, Dec 30 2014, Dmitry Gutov wrote:

> Suggestions for a better implementation are welcome. Should we mess
> around with `window-configuration-change-hook'?

My suggestion would be to stay away from window management as much as you
can because it's very difficult to get it right.

q followed by M-, usually brings back the original window configuration.
Based on that, maybe q should combine quit-window and
xref-pop-marker-stack (preferably recognizing whether q was pressed
multiple times).

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/31/2014 12:58 AM, Helmut Eller wrote:

> My suggestion would be to stay away from window management as much as you
> can because it's very difficult to get it right.

I agree it can be a minefield, so if anyone sees a case that's 
particularly difficult to handle this way, please let me know.

> q followed by M-, usually brings back the original window configuration.

Only it doesn't exactly work that way if there are more than two 
windows, because we open the xrefs' buffers in the "other window" in 
reference to the *xrefs* window, and that's not the original window.

We could try to always use the original window for that, but that would 
be more limiting (why not use different windows, if they are available?).

If someone really dislikes the current behavior, we can go that way, but 
for now I've installed some of the already mentioned minor tweaks 
(5496236583e22bce185650988a1ae675216e883e).

bug#19468: 25.0.50; UI inconveniences with M-.

[martin rudalics]

 >> My suggestion would be to stay away from window management as much as you
 >> can because it's very difficult to get it right.
 >
 > I agree it can be a minefield, so if anyone sees a case that's particularly difficult to handle this way, please let me know.
 >
 >> q followed by M-, usually brings back the original window configuration.

If and only if the selected window is the only window that changed.  The
idea of `quit-window' is to undo only changes made for this particular
window.  It should _never_ undo changes made to the rest of the window
configuration.

 > Only it doesn't exactly work that way if there are more than two windows, because we open the xrefs' buffers in the "other window" in reference to the *xrefs* window, and that's not the original window.

You could try to bind "q" to a function that invokes `quit-window' for
both windows (preferably in inverse order of how they were made).

martin

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 01/04/2015 11:55 AM, martin rudalics wrote:

> You could try to bind "q" to a function that invokes `quit-window' for
> both windows (preferably in inverse order of how they were made).

In all likelihood, we've displayed several buffers in succession in that 
other window. Or none. Doing `quit-window' there exactly once won't suffice.

So I guess this amounts to keeping a buffer-window history of everything 
that was opened from *xrefs*, and on `q', iterate through that list and, 
if the given window displays the given buffer, `quit-buffer' on it. Then 
clear the history and quit *xrefs*.

I kinda like that.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 12/30/2014 06:41 PM, Eli Zaretskii wrote:

> More importantly, it will tell the user what she needs to do, even if
> she doesn't use the mouse, because mouse-sensitive portions of Emacs
> display generally behave very similarly.

I've added some button-like interaction, but not exactly using buttons 
(largely because of bug#19500).

The interaction might be a bit unconventional by Emacs standards 
(mouse-2 works like C-o; mouse-1 works like RET), but see how it works 
for you. Reverting the bindings is not out of the question.

bug#19468: 25.0.50; UI inconveniences with M-.

[martin rudalics]

 > In all likelihood, we've displayed several buffers in succession in that other window. Or none. Doing `quit-window' there exactly once won't suffice.

It should, conceptually.

 > So I guess this amounts to keeping a buffer-window history of everything that was opened from *xrefs*, and on `q', iterate through that list and, if the given window displays the given buffer, `quit-buffer' on it. Then clear the history and quit *xrefs*.

We have two windows we want to quit simultaneously.  Any of them could
have been made specially for xrefing or have displayed some other buffer
before.  In the former case the window would have to be deleted in the
latter made to show the buffer it showed before xrefing.

The entire information what to do should be in the `quit-restore'
parameter of any of these windows.  If what I described above doesn't
work out automatically you might have to adjust the `quit-restore'
parameter manually.  But it should work automatically.

 > I kinda like that.

You shouldn't.

martin

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Mon, 05 Jan 2015 02:52:51 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 12/30/2014 06:41 PM, Eli Zaretskii wrote:
> 
> > More importantly, it will tell the user what she needs to do, even if
> > she doesn't use the mouse, because mouse-sensitive portions of Emacs
> > display generally behave very similarly.
> 
> I've added some button-like interaction, but not exactly using buttons 
> (largely because of bug#19500).
> 
> The interaction might be a bit unconventional by Emacs standards 
> (mouse-2 works like C-o; mouse-1 works like RET), but see how it works 
> for you. Reverting the bindings is not out of the question.

Looks OK; adding help-echo will make it even better.  Thanks.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 01/05/2015 10:08 AM, martin rudalics wrote:
>  > In all likelihood, we've displayed several buffers in succession in
> that other window. Or none. Doing `quit-window' there exactly once won't
> suffice.
>
> It should, conceptually.

One `quit-window' would switch to the buffer below it. If that one was 
also displayed by xref, we don't gain much at all.

And what if, like I already mentioned, xref hasn't displayed any xref 
definitions yet?

> We have two windows we want to quit simultaneously.

Quit windows or not, that's an implementation detail. We want to restore 
window configuration before xref, but only if the user hasn't tampered 
with it in the process.

> Any of them could
> have been made specially for xrefing

Not really: `xref--display-position' calls (display-buffer 
(current-buffer) t).

> The entire information what to do should be in the `quit-restore'
> parameter of any of these windows.  If what I described above doesn't
> work out automatically you might have to adjust the `quit-restore'
> parameter manually.  But it should work automatically.

What then, you would rather I save the original buffer displayed in the 
other window, and then changed the `quit-restore' parameter in that 
window before quitting? I could just as well do all the `bury-buffer' 
myself.

And that won't take care of the case when the window configuration is 
changed by anything other than xref.

I've pushed my implementation using `quit-window' to scratch/xref, 
please take a look. What would you do better?

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

So, prompted by the renewed discussion of 'xref', I took another look
at the current states of affairs with the UI, and found the following
issues:

 . xref-etags-mode:

   I tried this mode, and found a few places for improvement:

   . It's not available by default:

     emacs -Q
     M-x xref-et TAB => [No match]

     It should be autoloaded, IMO.  Perhaps there should also be a
     global minor mode, for those who'd like that.

   . It does nothing when point is on a unique symbol:

     M-. set_cursor_from_row RET
     Now, with point on its position under set_cursor_from_row, type
      M-. again => nothing(??!!) happens

     I guess it looks for more symbols matching set_cursor_from_row,
     finds out that this is the only one, and does nothing.  This is
     not useful.  It should at the very least say something like
     "set_cursor_from_row: this is the only match".  Bonus points for
     prompting for a symbol, like it does when there's no symbol at
     point, because I think this is more useful in this situation.

   . Some problems with key bindings in the *xref* buffer:

     . RET displays the candidate listed on the current line, but
       closes the window displaying *xref*, so it's not easy to try
       another candidate afterwards.  I think it would be more helpful
       to just switch to the window showing the definition, but leave
       the *xref* buffer shown.
     . You need to use the unconventional '.' and ',' to both move and
       display the definitions -- this should at least be mentioned in
       the initial echo-area message when *xref* is first displayed.
       (This was reported as a problem in the original report, but
       seems to be left unchanged.)

   . No replacement for tags-loop-continue:

     If I somehow let the window showing *xref* close (e.g., by typing
     RET, see above), what is the equivalent for tags-loop-continue,
     i.e. display the next candidate, without explicitly displaying
     the *xref* buffer again?  If there isn't one, please provide it,
     at least as part of xref-etags-mode.  (Other similar Emacs modes,
     like Compilation and Grep, do provide commands to move to the
     next item without first switching to the buffer that shows all
     the hits.)

 . The back-end issue:

   . Caveat: I couldn't find any documentation of back-ends, so
     perhaps I'm missing something important.  That being said...

   . I tried the ELisp back-end and found that it somehow affects the
     UI, so that the UI behaves differently than with the default
     etags back-end, when the user types something that is "complete,
     but not unique": when using the etags back-end, Emacs displays a
     list of candidates in the *xref* buffer, whereas with the
     elisp-mode back-end it shows the "complete" candidate, doesn't
     display *xref*, and doesn't insert the other candidates into
     *xref*.  Is this difference intended?  It's confusing, to say the
     least.

   . xref.el says:

     ;; For now, make the etags backend the default.
     (defvar xref-find-function #'etags-xref-find

     But what are the alternatives, if any?  I could only find
     something related in ada-mode and in elisp-mode.  This means
     that, for example, for C/C++ and Java, etags is the only
     available back-end, and this change is currently just a different
     UI wrapping the same basic functionality?  Is there any further
     development planned for the near future?

   . The doc string of xref-find-function mentions several variants of
     invoking the function, but there doesn't seem to be any way of
     controlling that when invoking the function interactively, is
     there?  I think it would be good to be able to lookup only the
     definitions or only the references of a symbol.

HTH

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>    . I tried the ELisp back-end and found that it somehow affects the
>      UI, so that the UI behaves differently than with the default
>      etags back-end, when the user types something that is "complete,
>      but not unique": when using the etags back-end, Emacs displays a
>      list of candidates in the *xref* buffer, whereas with the
>      elisp-mode back-end it shows the "complete" candidate, doesn't
>      display *xref*, and doesn't insert the other candidates into
>      *xref*.  Is this difference intended?  It's confusing, to say the
>      least.

I don't understand exactly the scenario you're talking about.  Can you
give a recipe?

>      But what are the alternatives, if any?  I could only find
>      something related in ada-mode and in elisp-mode.  This means
>      that, for example, for C/C++ and Java, etags is the only
>      available back-end, and this change is currently just a different
>      UI wrapping the same basic functionality?  Is there any further
>      development planned for the near future?

There should very much be other backends on the way, e.g. using Semantic
(for C/C++), SLIME (for CL), ...

>    . The doc string of xref-find-function mentions several variants of
>      invoking the function, but there doesn't seem to be any way of
>      controlling that when invoking the function interactively, is
>      there?  I think it would be good to be able to lookup only the
>      definitions or only the references of a symbol.

Indeed, the current UI does not offer access to all features of the API.
Improvements welcome,


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 00:30:41 -0400
> 
> >    . I tried the ELisp back-end and found that it somehow affects the
> >      UI, so that the UI behaves differently than with the default
> >      etags back-end, when the user types something that is "complete,
> >      but not unique": when using the etags back-end, Emacs displays a
> >      list of candidates in the *xref* buffer, whereas with the
> >      elisp-mode back-end it shows the "complete" candidate, doesn't
> >      display *xref*, and doesn't insert the other candidates into
> >      *xref*.  Is this difference intended?  It's confusing, to say the
> >      least.
> 
> I don't understand exactly the scenario you're talking about.  Can you
> give a recipe?

Yes:

  emacs -Q
  C-x C-f lisp/simple.el RET
  M-. find-tag RET

This puts you at the first line of find-tag, without showing the other
possible symbols whose names contain "find-tag" as their substring.
If you want the other candidates, you need to type TAB instead of RET,
and then select the one you want via the usual completion facilities.

By contrast, this:

  emacs -Q
  C-x C-f lisp/simple.el RET
  M-x load-library RET xref RET
  M-x xref-etags-mode RET
  M-. find-tag RET

does NOT show the definition of find-tag, but instead opens an *xref*
buffer with possible matches, and expects you to pick one of them (and
btw the defun I'm probably after is not the first or second hit there;
looks like the candidates are arranged in the alphabetical order of
the *.el files).

I guess the elisp-mode back-end returns just one candidate, whereas
the etags back-end returns a list.  But it's confusing to have such
evident differences just because you changed the back-end, I think.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 00:30:41 -0400
> 
> >      But what are the alternatives, if any?  I could only find
> >      something related in ada-mode and in elisp-mode.  This means
> >      that, for example, for C/C++ and Java, etags is the only
> >      available back-end, and this change is currently just a different
> >      UI wrapping the same basic functionality?  Is there any further
> >      development planned for the near future?
> 
> There should very much be other backends on the way, e.g. using Semantic
> (for C/C++), SLIME (for CL), ...

I certainly hope that at least the Semantic one materializes soon
enough, otherwise it sounds like all this move to xref was for the
benefit of unbundled packages, and users of Emacs are just punished by
having to learn a new UI for no real advantage.

> >    . The doc string of xref-find-function mentions several variants of
> >      invoking the function, but there doesn't seem to be any way of
> >      controlling that when invoking the function interactively, is
> >      there?  I think it would be good to be able to lookup only the
> >      definitions or only the references of a symbol.
> 
> Indeed, the current UI does not offer access to all features of the API.
> Improvements welcome,

Not from me: I've been told in this discussion that I don't understand
the new UI, don't appreciate its many virtues, and generally am no
more than an obstacle on the way of progress.  So I really don't think
I should mess with this UI.

That said, the usual way in Emacs to provide minor variants of a
command is via special values of the prefix argument, like, for
example, "C-x C-s" does.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

> I certainly hope that at least the Semantic one materializes soon
> enough, otherwise it sounds like all this move to xref was for the
> benefit of unbundled packages,

IMO one of the main tasks of the maintainers is to provide
infrastructure that consolidates the features that are duplicated out
there in many different packages, all mutually slightly incompatible.

So the usefulness does not directly depend on whether that feature is
used by bundled or unbundled packages.  Emacs thrives in large part
thanks to all the unbundled packages out there and we shouldn't treat
them as second-rate citizens.

[ Of course, I much prefer when a package's copyright is assigned and when
it's either in GNU ELPA or in Emacs proper, especially because it makes
it easier to incorporate its features into the core so as to expand the
general infrastructure and let other packages build on it.  ]

> and users of Emacs are just punished by having to learn a new UI for
> no real advantage.

I like the new UI, FWIW.  But clearly, we need to work to make it better
for those users who miss some aspects of the old UI.

BTW, w.r.t the use of etags, what do you think of the patch below, which
lets the elisp-xref functionality fallback on etags for functions/vars
that aren't currently bound?
[ Clearly, the patch doesn't work as is, since it only works once xref
  is loaded, but I hope you see what this is intended to do.  ]

> That said, the usual way in Emacs to provide minor variants of a
> command is via special values of the prefix argument, like, for
> example, "C-x C-s" does.

The different ways to call xref-find-function are to distinguish
jumping to the definition or to "all uses", and currently few backends
support those features, I don't think we have a clear idea yet of how
they should be presented to the user, and there are too many variants to
collapse them all into C-u.  So they'd probably be provided via
different commands instead of all being accessed via M-.


        Stefan


diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index f289068..04bf49d 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -235,9 +235,9 @@ Blank lines separate paragraphs.  Semicolons start comments.
   (setq imenu-case-fold-search nil)
   (add-function :before-until (local 'eldoc-documentation-function)
                 #'elisp-eldoc-documentation-function)
-  (setq-local xref-find-function #'elisp-xref-find)
-  (setq-local xref-identifier-completion-table-function
-              #'elisp--xref-identifier-completion-table)
+  (add-function :before-until (local 'xref-find-function) #'elisp-xref-find)
+  (add-function :before-until (local 'xref-identifier-completion-table-function)
+                #'elisp--xref-identifier-completion-table)
   (add-hook 'completion-at-point-functions
             #'elisp-completion-at-point nil 'local))
 

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> I don't understand exactly the scenario you're talking about.  Can you
>> give a recipe?
>
> Yes:
>
>   emacs -Q
>   C-x C-f lisp/simple.el RET
>   M-. find-tag RET
>
> This puts you at the first line of find-tag, without showing the other
> possible symbols whose names contain "find-tag" as their substring.

Yes, that's the expected behavior, and matches the behavior of Emacs-24.
Is that a behavior you like or a behavior you dislike?

> By contrast, this:
>
>   emacs -Q
>   C-x C-f lisp/simple.el RET
>   M-x load-library RET xref RET
>   M-x xref-etags-mode RET
>   M-. find-tag RET
>
> does NOT show the definition of find-tag, but instead opens an *xref*
> buffer with possible matches, and expects you to pick one of them (and

Indeed, one of the differences in the new UI is that we don't have the
"C-u M-. lets you cycle among matches" any more.  Instead we pop up an
*xref* buffer where you can choose the match that interests you.  I find
this easier (and generally faster) to use.

This is an issue that only affects the UI (not the API and hence not the
backends), so we can easily fix it by providing either different UIs, or
some config var, or ...

> btw the defun I'm probably after is not the first or second hit there;
> looks like the candidates are arranged in the alphabetical order of
> the *.el files).

Indeed, the ordering looks incorrect.  This is a bug.

> I guess the elisp-mode back-end returns just one candidate, whereas
> the etags back-end returns a list.

Exactly.

> But it's confusing to have such evident differences just because you
> changed the back-end, I think.

But popping up the *xref* buffer when there's only one element in it
doesn't make much sense I think.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: dgutov@yandex.ru, 19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 13:21:07 -0400
> 
> > I certainly hope that at least the Semantic one materializes soon
> > enough, otherwise it sounds like all this move to xref was for the
> > benefit of unbundled packages,
> 
> IMO one of the main tasks of the maintainers is to provide
> infrastructure that consolidates the features that are duplicated out
> there in many different packages, all mutually slightly incompatible.
> 
> So the usefulness does not directly depend on whether that feature is
> used by bundled or unbundled packages.  Emacs thrives in large part
> thanks to all the unbundled packages out there and we shouldn't treat
> them as second-rate citizens.

We shouldn't treat the Emacs core as second-rate citizen, either.  A
feature as central to code development and Emacs in general as finding
definitions and references of symbols cannot thrive only (or mainly)
outside the core, IMO.

> > and users of Emacs are just punished by having to learn a new UI for
> > no real advantage.
> 
> I like the new UI, FWIW.

Where did you see me say I dislike it?  I'm just saying that learning
a new UI for the sake of a new UI is a waste.  I _am_ prepared to
learn a new UI if it brings new useful functionality with it.

> BTW, w.r.t the use of etags, what do you think of the patch below, which
> lets the elisp-xref functionality fallback on etags for functions/vars
> that aren't currently bound?

What's it supposed to do?

Anyway, I once again think that if we will mostly "fall back on etags"
every time the new back-ends are somehow imperfect, then what exactly
did we gain with this move?  I heard more than once in recent
discussions that the etags back-end is less than perfect, to put it
mildly; after hearing this so much, I wonder how it makes sense to
fall back on it all the time.  If it's not good, let's replace it with
something better, not fall back on it.

> > That said, the usual way in Emacs to provide minor variants of a
> > command is via special values of the prefix argument, like, for
> > example, "C-x C-s" does.
> 
> The different ways to call xref-find-function are to distinguish
> jumping to the definition or to "all uses", and currently few backends
> support those features

Sounds like another feature that was planned, but never implemented.
Once again, why did we switch to xref if so much of its promise is yet
to come?

> I don't think we have a clear idea yet of how they should be
> presented to the user

There's prior art out there in other IDEs, we could start by learning
from them.

> and there are too many variants to collapse them all into C-u.

"C-x C-s" manages to provide no less than 4 variants with C-u, so I
see no problem to use it with this command.  That said, ...

> So they'd probably be provided via different commands instead of all
> being accessed via M-.

... I won't mind this alternative, either, although we don't have too
many keys left to use for those other commands.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: dgutov@yandex.ru, 19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 13:35:25 -0400
> 
> >> I don't understand exactly the scenario you're talking about.  Can you
> >> give a recipe?
> >
> > Yes:
> >
> >   emacs -Q
> >   C-x C-f lisp/simple.el RET
> >   M-. find-tag RET
> >
> > This puts you at the first line of find-tag, without showing the other
> > possible symbols whose names contain "find-tag" as their substring.
> 
> Yes, that's the expected behavior, and matches the behavior of Emacs-24.

Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
moreover, if you try "C-u M-.", you get prompted for the symbol again,
and if you type the same one, you get nowhere.  The other matches are
only available via completion, see below.

> Is that a behavior you like or a behavior you dislike?

It's surprising, since I expected to see the *xref* buffer, as I
understood this is the core of the new UI.  But it sounds like the
back-end can easily affect the UI in radical ways, which I think is
not a good thing: the entire idea of the back-end means to me that the
front-end is not affected, UI-wise.

> > By contrast, this:
> >
> >   emacs -Q
> >   C-x C-f lisp/simple.el RET
> >   M-x load-library RET xref RET
> >   M-x xref-etags-mode RET
> >   M-. find-tag RET
> >
> > does NOT show the definition of find-tag, but instead opens an *xref*
> > buffer with possible matches, and expects you to pick one of them (and
> 
> Indeed, one of the differences in the new UI is that we don't have the
> "C-u M-. lets you cycle among matches" any more.  Instead we pop up an
> *xref* buffer where you can choose the match that interests you.  I find
> this easier (and generally faster) to use.

OK, but then why aren't we shown *xref* in the first case?

> > But it's confusing to have such evident differences just because you
> > changed the back-end, I think.
> 
> But popping up the *xref* buffer when there's only one element in it
> doesn't make much sense I think.

Oh, but there shouldn't be only one element: you will see the others
if you type TAB instead of RET.  IOW, the elisp-mode back-end does
know about the other candidates, it just doesn't show them in *xref*.
It somehow decided for me that I want only the exact match.

And btw, there is another confusing difference between these back-ends
that somehow bubbles up to the UI level: the elisp-mode back-end only
supports finding multiple matches via completion, and completion, at
least by default, only finds candidates whose leading substring
matches what you type.  By contrast, the *xref* buffer popped when the
etags back-end is in use shows substring matches _anywhere_ in the
symbol names, so it shows much more candidates.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> BTW, w.r.t the use of etags, what do you think of the patch below, which
>> lets the elisp-xref functionality fallback on etags for functions/vars
>> that aren't currently bound?
> What's it supposed to do?

Let you find the definition of a function that's not defined yet, thanks
to the use of the TAGS database.

> Anyway, I once again think that if we will mostly "fall back on etags"
> every time the new back-ends are somehow imperfect, then what exactly
> did we gain with this move?

I just suggest it to cover those cases where the user likes the old
etags behavior.  I personally preferred using C-h f until now over
etags's M-. because it was more reliable, so I find the new M-. to be
a good change and don't need to fallback to etags's backend.

> ... I won't mind this alternative, either, although we don't have too
> many keys left to use for those other commands.

They don't have to be single-key bindings.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

> Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
> moreover, if you try "C-u M-.", you get prompted for the symbol again,
> and if you type the same one, you get nowhere.  The other matches are
> only available via completion, see below.

Maybe we should write an xref-old-ui-mode.
It would make M-. always prompt and it would always jump to the first
entry and then let you use C-u M-. to jump to the next match.

>> But popping up the *xref* buffer when there's only one element in it
>> doesn't make much sense I think.
> Oh, but there shouldn't be only one element: you will see the others
> if you type TAB instead of RET.

No: there's only one definition of `find-tag' so Elisp's xref backend
just returns that one and that's it.

> IOW, the elisp-mode back-end does know about the other candidates, it
> just doesn't show them in *xref*.

None of the others are definitions of `find-tag'.

Arguably, the find-tag advice in org-ctags.el could be offered as well,
if org-ctags.el happens to be loaded (tho currently we could only do
that for an advice added via advice-add rather than via defadvice since
defadvice does not keep track of the origin).

> It somehow decided for me that I want only the exact match.

It's rather that the old etags interface was not sure enough of itself,
so it offered anything that had "find-tag" as a substring somewhere,
i.e. lots and lots of false positives.

> And btw, there is another confusing difference between these back-ends
> that somehow bubbles up to the UI level: the elisp-mode back-end only
> supports finding multiple matches via completion,

No.  `semantic-symref-find-tags-by-name' is not a match for "show me the
definition of find-tag" and neither is "find-tag-tag".  The problem here
is that the current etags xref backend returns confusingly many
so-called "matches" which are really irrelevant and should only be
included in the completions.

There's a difference between:

  I know the name, I need to find out its definition.

and

  I remember the name includes something like "find-tag"

The old code did not distinguish between the two, but the new code does.
I think that's an important feature of the new code in this respect.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 07:30 AM, Stefan Monnier wrote:

> There should very much be other backends on the way, e.g. using Semantic
> (for C/C++), SLIME (for CL), ...

Both CIDER and SLIME look like a good candidate among third-party 
packages. I also intend to use it in Robe (which can't be added to GNU 
ELPA because of inf-ruby dependency; it's also begging for a rewrite...)

>>     . The doc string of xref-find-function mentions several variants of
>>       invoking the function, but there doesn't seem to be any way of
>>       controlling that when invoking the function interactively, is
>>       there?  I think it would be good to be able to lookup only the
>>       definitions or only the references of a symbol.

`M-.' only looks up definitions. Or at least, that's the .

> Indeed, the current UI does not offer access to all features of the API.

There is M-x xref-find-references. Neither of the two current backends 
implements it, though (but I know SLIME has such a feature, and CEDET 
probably has something comparable). We could bind it to `C-M-?.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 06:07 PM, Eli Zaretskii wrote:

> This puts you at the first line of find-tag, without showing the other
> possible symbols whose names contain "find-tag" as their substring.
> If you want the other candidates, you need to type TAB instead of RET,
> and then select the one you want via the usual completion facilities.

Yes, if you want all matches, try `xref-find-apropos' (bound to 
`C-M-.').  Usually I prefer the new, more precise behavior of `M-.'.

> I guess the elisp-mode back-end returns just one candidate, whereas
> the etags back-end returns a list.  But it's confusing to have such
> evident differences just because you changed the back-end, I think.

etags showing all matches is, in general, the limitation of the tags 
file format. But if someone submits a patch making it more precise 
without creating false negatives, I'm all for it.

And hey, you changed the back-end. It uses a different mechanism under 
the covers. *Of course* you're going to get different results, in many 
cases.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 06:13 PM, Eli Zaretskii wrote:

> I certainly hope that at least the Semantic one materializes soon
> enough, otherwise it sounds like all this move to xref was for the
> benefit of unbundled packages, and users of Emacs are just punished by
> having to learn a new UI for no real advantage.

It's probably up to CEDET developers.

But the UI itself is one of the benefits. If the old one was considered 
superior, the new API could have been introduced without really changing 
the UI.

>>>     . The doc string of xref-find-function mentions several variants of
>>>       invoking the function, but there doesn't seem to be any way of
>>>       controlling that when invoking the function interactively, is
>>>       there?  I think it would be good to be able to lookup only the
>>>       definitions or only the references of a symbol.

There seems some confusion here: you do now invoke xref-find-function. 
It's not a function anyway, it's a variable, which different backends 
can set to implement the backend interface.

You invoke different commands from the xref package. Try typing `M-x 
xref-find- TAB'.

> Not from me: I've been told in this discussion that I don't understand
> the new UI, don't appreciate its many virtues, and generally am no
> more than an obstacle on the way of progress.

I really don't remember stating that in this discussion.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 08:21 PM, Stefan Monnier wrote:

> The different ways to call xref-find-function are to distinguish
> jumping to the definition or to "all uses", and currently few backends
> support those features, I don't think we have a clear idea yet of how
> they should be presented to the user, and there are too many variants to
> collapse them all into C-u.  So they'd probably be provided via
> different commands instead of all being accessed via M-.

M-x xref-find-references, for starters.

> -  (setq-local xref-find-function #'elisp-xref-find)
> -  (setq-local xref-identifier-completion-table-function
> -              #'elisp--xref-identifier-completion-table)
> +  (add-function :before-until (local 'xref-find-function) #'elisp-xref-find)
> +  (add-function :before-until (local 'xref-identifier-completion-table-function)
> +                #'elisp--xref-identifier-completion-table)

This doesn't look right:

- When I try to jump to a symbol, and there isn't one with that name, I 
want to see the message "no definitions found", not a "Visit tags table" 
prompt.

- If I do have a tags table loaded, it's likely for a different project 
than Emacs. This is most likely true for the majority of our users.

I think Eli (and everyone like-minded) should advise elisp-xref-find and 
xref-identifier-completion-table-function instead, inside their init 
script, like suggested by Daniel in another thread.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 10:15 PM, Eli Zaretskii wrote:

> We shouldn't treat the Emacs core as second-rate citizen, either.  A
> feature as central to code development and Emacs in general as finding
> definitions and references of symbols cannot thrive only (or mainly)
> outside the core, IMO.

So? What are you doing to help with that? Are you asking anyone 
currently participating in this thread to work on CEDET?

> Where did you see me say I dislike it?  I'm just saying that learning
> a new UI for the sake of a new UI is a waste.  I _am_ prepared to
> learn a new UI if it brings new useful functionality with it.

The new UI itself brings tangible improvements by itself, as well as a 
unified interface that can (and hopefully will) be used by different 
projects. I don't know how you don't see the value in that.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 10:26 PM, Eli Zaretskii wrote:

> And btw, there is another confusing difference between these back-ends
> that somehow bubbles up to the UI level: the elisp-mode back-end only
> supports finding multiple matches via completion, and completion, at
> least by default, only finds candidates whose leading substring
> matches what you type.

The substring vs. prefix matching is an interesting distinction. Our 
completion tables only work with prefix matching.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/27/2015 08:35 PM, Stefan Monnier wrote:

>> btw the defun I'm probably after is not the first or second hit there;
>> looks like the candidates are arranged in the alphabetical order of
>> the *.el files).
>
> Indeed, the ordering looks incorrect.  This is a bug.

The idea here is to group candidates belonging to different files 
together. Sorting the files alphabetically after that is the first 
natural choice, but how else to do it better?

- Should xref always try to preserve the order in which xrefs are 
returned? Some backends might not mean anything by it.

- Suppose one xref in a.el is returned before all xrefs in b.el. And 
another xref in a.el is returned after. In what order will we display 
a.el and b.el?

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 12:54 AM, Stefan Monnier wrote:

> Maybe we should write an xref-old-ui-mode.
> It would make M-. always prompt and it would always jump to the first
> entry and then let you use C-u M-. to jump to the next match.

That's doable, but it's not high on my priorities list. In case anyone's 
wondering.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

> - When I try to jump to a symbol, and there isn't one with that name, I want
>   to see the message "no definitions found", not a "Visit tags table" prompt.

I tend to agree.

> - If I do have a tags table loaded, it's likely for a different project than
>   Emacs.  This is most likely true for the majority of our users.

I really think the etags backend should only return data when the TAGS
file is in one of the parents of the current directory.

And it should also be able to keep several independent TAGS tables
opened for different projects in different directory trees.

> I think Eli (and everyone like-minded) should advise elisp-xref-find and
> xref-identifier-completion-table-function instead, inside their init script,
> like suggested by Daniel in another thread.

It seems a common enough request that we should accommodate
them better. At the very least provide a minor mode which sets up
such advice.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 01:54 AM, Stefan Monnier wrote:

> I really think the etags backend should only return data when the TAGS
> file is in one of the parents of the current directory.

That goes against what Eli asked for in the past, though.

And unless loading of the TAGS file happens automatically (another 
change), this will do away with the convenience of the automatic 
suggestion when I do want to use etags.

> It seems a common enough request that we should accommodate
> them better. At the very least provide a minor mode which sets up
> such advice.

But how common is it, really? I've only seen one person ask for it so 
far. Surely the 5 or so people who actually use tags with Elisp are a 
distinct minority compared to the whole Emacs user base.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> And btw, there is another confusing difference between these back-ends
>> that somehow bubbles up to the UI level: the elisp-mode back-end only
>> supports finding multiple matches via completion, and completion, at
>> least by default, only finds candidates whose leading substring
>> matches what you type.
> The substring vs. prefix matching is an interesting distinction. Our
> completion tables only work with prefix matching.

While the underlying completion tables indeed only provide prefix
completion, the normal completion UI provides the `substring'
completion style.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>>> btw the defun I'm probably after is not the first or second hit there;
>>> looks like the candidates are arranged in the alphabetical order of
>>> the *.el files).
>> Indeed, the ordering looks incorrect.  This is a bug.
> The idea here is to group candidates belonging to different
> files together.

That makes a lot of sense when the candidates are equally likely to be
The One that the user's looking for.  In the case of etags I think the
problem comes from the substring matching it uses.
Can't etags do better than substring matching and only return matches
that actually have the requested name (and maybe fallback on substring
matching if there's no exact match)?


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 02:15 AM, Stefan Monnier wrote:

> While the underlying completion tables indeed only provide prefix
> completion, the normal completion UI provides the `substring'
> completion style.

We can't use that in minibuffer though, right?

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/26/2015 05:56 PM, Eli Zaretskii wrote:

>       emacs -Q
>       M-x xref-et TAB => [No match]
>
>       It should be autoloaded, IMO.  Perhaps there should also be a
>       global minor mode, for those who'd like that.

I don't know if we want to settle on that solution, actually. Have you 
seen this thread?

http://lists.gnu.org/archive/html/emacs-devel/2015-04/msg01236.html

That's not so easy to wrap in a global minor mode. I'm not even sure a 
minor mode would be a good approach here at all.

However, it's a lot more flexible than using xref-etags-mode, especially 
with the changes to help buffers proposed by Daniel.

>     . It does nothing when point is on a unique symbol:
>
>       M-. set_cursor_from_row RET
>       Now, with point on its position under set_cursor_from_row, type
>        M-. again => nothing(??!!) happens
>
>       I guess it looks for more symbols matching set_cursor_from_row,
>       finds out that this is the only one, and does nothing.  This is
>       not useful.  It should at the very least say something like
>       "set_cursor_from_row: this is the only match".

`M-x find-tag' behaves exactly the same: even when the result is that we 
don't move anywhere (for the same reason), there's no helpful message. 
Just "Mark set" in the echo area.

I don't know if adding a new message in this case will be too helpful, 
but you're welcome to suggest the wording.

 >   Bonus points for
 >       prompting for a symbol, like it does when there's no symbol at
 >       point, because I think this is more useful in this situation.

I'd take a look at the patch.

>     . Some problems with key bindings in the *xref* buffer:
>
>       . RET displays the candidate listed on the current line, but
>         closes the window displaying *xref*, so it's not easy to try
>         another candidate afterwards.

There's also the `C-o' binding, which displays the xref, but keeps the 
current focus.

 >  I think it would be more helpful
 >         to just switch to the window showing the definition, but leave
 >         the *xref* buffer shown.

Some packages take this approach; I don't like it. But this should be 
easy to implement with a user option that would slightly change the 
behavior of `xref-goto-xref'. Care to take a stab at it?

>       . You need to use the unconventional '.' and ',' to both move and
>         display the definitions -- this should at least be mentioned in
>         the initial echo-area message when *xref* is first displayed.
>         (This was reported as a problem in the original report, but
>         seems to be left unchanged.)

Should those be `n' and `p' instead, by default? I've found myself using 
these bindings very rarely anyway, and `n' is still very close.

>      (Other similar Emacs modes,
>       like Compilation and Grep, do provide commands to move to the
>       next item without first switching to the buffer that shows all
>       the hits.)

Compilation and Grep both use M-g M-n/M-g M-p, and the 
next-error-function variable. Do you want xref to reuse it?

Would you be comfortable with forgetting the current list of errors 
after navigating somewhere with xref?

>       But what are the alternatives, if any?  I could only find
>       something related in ada-mode and in elisp-mode.  This means
>       that, for example, for C/C++ and Java, etags is the only
>       available back-end, and this change is currently just a different
>       UI wrapping the same basic functionality?  Is there any further
>       development planned for the near future?

There's also ggtags in GNU ELPA. I'm sure it could provide some 
integration, too.

bug#19468: 25.0.50; UI inconveniences with M-.

[Drew Adams]

> > - When I try to jump to a symbol, and there isn't one with that
> >   name, I want to see the message "no definitions found", not a
> >   "Visit tags table" prompt.
> 
> I tend to agree.

IIUC, that seems a bit ironic.

You took the opposite point of view when I said the same thing
about `completion-styles' that you seem to be saying here.

I made just that argument: let users know when a given completion
method finds no matches, and let them choose whether to go on to
trying another method.

This seems similar - the difference being that instead of
different ways of matching completion candidates you are dealing
with different "backends" their candidate sets.

Giving users feedback such as "No definitions found" and "No
completions", instead of blindly going on to look for other
hits using other means, is helpful and gives them more control.

At the very least it should be an option: let them decide
whether to go on automatically to other attempts.

Which is also the argument I made wrt `completion-styles':
let users choose which styles should chain automatically to
which others, by grouping them.  No group(s) means no automatic
chaining.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 02:19 AM, Stefan Monnier wrote:

>> The idea here is to group candidates belonging to different
>> files together.

    ^^^ to the same file, sorry.

> That makes a lot of sense when the candidates are equally likely to be
> The One that the user's looking for.  In the case of etags I think the
> problem comes from the substring matching it uses.

etags puts some effort into returning the candidates in the most 
plausible way. Should we ignore that? etags--xref-find-definitions tries 
to honor it, at least according to the comment.

Further, xref-find-apropos will return a lot of candidates, and that 
will be what the user wants.

> Can't etags do better than substring matching and only return matches
> that actually have the requested name (and maybe fallback on substring
> matching if there's no exact match)?

I don't know if this problem is solvable in a generic way.

How reliable is tag-exact-match-p? We could, for instance, filter out 
all non-exact matches when we're just looking for definitions.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> I really think the etags backend should only return data when the TAGS
>> file is in one of the parents of the current directory.
> That goes against what Eli asked for in the past, though.

Yes, we've been through that, indeed.  But it can be accomodated by
requiring the user to explicitly say that some TAGS file covers "these
and these extra directory trees" or to link TAGS file together.

> And unless loading of the TAGS file happens automatically (another change),
> this will do away with the convenience of the automatic suggestion when I do
> want to use etags.

The automatic suggestion could be restricted to only come about when
there is a TAGS file in one of the parent dirs.

> But how common is it, really? I've only seen one person ask for it so
> far.  Surely the 5 or so people who actually use tags with Elisp are
> a distinct minority compared to the whole Emacs user base.

Could be.  But etags has been around for a long time and was the only
"standard feature" of the kind until very recently in Emacs, so I think
it's fair to assume that there are more such users out there.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> While the underlying completion tables indeed only provide prefix
>> completion, the normal completion UI provides the `substring'
>> completion style.
> We can't use that in minibuffer though, right?

Of course we can (and we do for example in C-x b).


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> That makes a lot of sense when the candidates are equally likely to be
>> The One that the user's looking for.  In the case of etags I think the
>> problem comes from the substring matching it uses.
> etags puts some effort into returning the candidates in the most plausible
> way.

Indeed.

> Should we ignore that?

IIUC, in the current xref/etags code, it seems we do ignore that,
apparently because of the reordering.

> Further, xref-find-apropos will return a lot of candidates, and that will be
> what the user wants.

For xref-find-apropos that's fine.  But not for M-.

> How reliable is tag-exact-match-p?

Don't know.

> We could, for instance, filter out all
> non-exact matches when we're just looking for definitions.

I think that would make sense, yes.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 04:26 AM, Stefan Monnier wrote:

> Of course we can (and we do for example in C-x b).

Is that because in that case the collection is not a function?

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 04:25 AM, Stefan Monnier wrote:

> Yes, we've been through that, indeed.  But it can be accomodated by
> requiring the user to explicitly say that some TAGS file covers "these
> and these extra directory trees" or to link TAGS file together.

Well, that's a bigger project than the 4-line patch you've suggested, 
and it requires manual intervention from the user. You suggested it 
before too, and I don't remember Eli being very enthusiastic.

> The automatic suggestion could be restricted to only come about when
> there is a TAGS file in one of the parent dirs.

That won't help when I'm inside Emacs sources; I still don't want to use 
tags for Elisp.

> Could be.  But etags has been around for a long time and was the only
> "standard feature" of the kind until very recently in Emacs, so I think
> it's fair to assume that there are more such users out there.

etags might be popular (so some sort a general solution might be 
welcome), but that doesn't mean it's used with Elisp a lot.

For instance, if we consider Debian-like distributions, 
https://packages.debian.org/jessie/all/emacs24-el/filelist does not 
include the TAGS file. I don't think the C sources are generally 
included in the binary distributions.

And if we're just talking about Emacs developers, I wonder why nobody 
else spoke up on emacs-devel or in this bug yet.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> Of course we can (and we do for example in C-x b).
> Is that because in that case the collection is not a function?

No.  As a matter of fact, in that case the collection *is* a function
(which removes the current buffer from the available choices).

The reason is because that `completion-metadata' says that the
completion category is `buffer' and because completion-category-defaults
says that for `buffer' completion, we should use the `substring' style
by default.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> Yes, we've been through that, indeed.  But it can be accomodated by
>> requiring the user to explicitly say that some TAGS file covers "these
>> and these extra directory trees" or to link TAGS file together.
> Well, that's a bigger project than the 4-line patch you've suggested,

Indeed, and it's largely unrelated since it would be a patch to the
xref/etags backend and wouldn't touch elisp-mode.el.

> and it requires manual intervention from the user.

More manual interventions in some cases, and less in others.

>> The automatic suggestion could be restricted to only come about when
>> there is a TAGS file in one of the parent dirs.
> That won't help when I'm inside Emacs sources; I still don't want to use
> tags for Elisp.

As mentioned, this suggestion is not directly related to other backends,
so whether xref/elisp completely shadows xref/etags or not is orthogonal.
BTW: Do you have a lisp/TAGS file?  If so, why?


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: dgutov@yandex.ru, 19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 17:54:30 -0400
> 
> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
> > and if you type the same one, you get nowhere.  The other matches are
> > only available via completion, see below.
> 
> Maybe we should write an xref-old-ui-mode.

I didn't ask for the old UI.  (If I want it, I can still have it,
since the etags.el commands are still available.)  I'm okay with the
new UI.  This bug report is about the new UI, and its goal is to make
the new UI better.

> It would make M-. always prompt and it would always jump to the first
> entry and then let you use C-u M-. to jump to the next match.

That's not what I meant by the above.  It was you who mentioned the
old UI, not me.  What I wanted to point out in response was that the
old UI was consistent and complete: it always displayed the first/next
match and provided a command to go to the next/previous one.  By
contrast, the new UI is inconsistent: with some back-ends it shows the
list of matches and allows to navigate it, with others it shows only
the first match and does not give any way to get the next match.

> >> But popping up the *xref* buffer when there's only one element in it
> >> doesn't make much sense I think.
> > Oh, but there shouldn't be only one element: you will see the others
> > if you type TAB instead of RET.
> 
> No: there's only one definition of `find-tag' so Elisp's xref backend
> just returns that one and that's it.

There's more than one matching definition, see below.

But that's largely immaterial: this bug report is not about the
back-end, it's about the UI.  The UI should be independent of the
back-end, otherwise the users will be confused when they switch
between languages.  (If it turns out that some back-ends need to be
augmented so that they allow the front-end to present similar UI for
the same query, then those back-ends should be enhanced.)

> > IOW, the elisp-mode back-end does know about the other candidates, it
> > just doesn't show them in *xref*.
> 
> None of the others are definitions of `find-tag'.

At least one is, the one in org-ctags.  And I can make a case for the
others as well, because no one said partial matches found by etags are
not legitimate, even if you personally don't consider them to be.

> Arguably, the find-tag advice in org-ctags.el could be offered as well,
> if org-ctags.el happens to be loaded

??? Why should the xref matches depend on what is and isn't loaded?
That would make xref exactly equivalent to "M-x apropos", which means
this mode of operation would make xref entirely redundant.  To say
nothing of the fact that this doesn't scale to any language except
ELisp.

We could offer this mode as an optional feature, but it certainly
shouldn't be the default.  (One of my worst annoyances is to type a
C-h command and be presented with "[No match]", because some package
is not loaded or some function is not available in the Emacs
configuration I happen to be using.)

> > It somehow decided for me that I want only the exact match.
> 
> It's rather that the old etags interface was not sure enough of itself,
> so it offered anything that had "find-tag" as a substring somewhere,
> i.e. lots and lots of false positives.

That's one way of looking at it.  Another is to say that etags gave
you more information and thus allowed to find more loose matches,
which is helpful when your memory is failing you.

But in the context of this bug report, that, too, is immaterial: if we
think the etags back-end gives too much information, by all means
let's filter it before presenting the matches.  Bonus points for
making the filtering optional, since some people might like that, and
in some situations even you might need it.

The important point for the purposes of this bug report is that the
stuff presented by the UI for the same query does not differ so
radically when you switch the back-end.

> > And btw, there is another confusing difference between these back-ends
> > that somehow bubbles up to the UI level: the elisp-mode back-end only
> > supports finding multiple matches via completion,
> 
> No.  `semantic-symref-find-tags-by-name' is not a match for "show me the
> definition of find-tag" and neither is "find-tag-tag". The problem here
> is that the current etags xref backend returns confusingly many
> so-called "matches" which are really irrelevant and should only be
> included in the completions.

Then let's fix the results we display with the etags back-end to show
only the relevant ones.  Please don't forget that currently, and
probably for the foreseeable future, the etags back-end is the only
one we have for most languages, so the things you dislike about it are
pretty much in our face for months, if not years, to come.  If we
think this back-end "needs work", let's do that now.  Saying that the
back-end returns a confusingly large amount of bogus matches, and then
switching to a UI that assumes much smarter back-ends (which don't
really exist) makes very little sense to me.

Whatever we do, let's make the UI more consistent and independent of
the back-ends.  Otherwise, we have a significant usability problem
waiting to happen.

> There's a difference between:
> 
>   I know the name, I need to find out its definition.
> 
> and
> 
>   I remember the name includes something like "find-tag"
> 
> The old code did not distinguish between the two, but the new code does.

The old code attempted to support both use cases, by showing the exact
match(es) first, followed by less likely ones.  We do similar things
in other commands.  The advantages of such a method are that (1) you
don't need to second-guess the user, (2) you provide only one command
for the user to remember, and (3) if the user only _thinks_ she knows
the name, but really doesn't, she can find the information without
having to invoke another command.

> I think that's an important feature of the new code in this respect.

But it is incomplete without a means to get to the other possible
matches in this case.

To summarize my points in this sub-thread:

  . the UI should depend much less on the back-end, ideally not at all

  . there should be a way to go to the next/previous match if the
    *xref* buffer is not displayed (or not created in the first
    place); if this happens because there's only one match, we should
    say so explicitly

  . when there are more than one possible match, the UI should behave
    similarly, i.e. display the *xref* buffer; when there's only one
    match, it should _never_ display *xref*, and should display an
    indication of the fact that there's only one match

  . if the criteria for filtering of the matches are very different
    between the possible back-ends, there should be some agreed-upon
    uniform default that returns roughly the same number of matches
    with all back-ends, and the rest should be controlled by user
    options

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 01:09:40 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>     This puts you at the first line of find-tag, without showing the other
>     possible symbols whose names contain "find-tag" as their substring.
>     If you want the other candidates, you need to type TAB instead of RET,
>     and then select the one you want via the usual completion facilities.
> 
> Yes, if you want all matches, try `xref-find-apropos' (bound to `C-M-.').

Some would say it's a disadvantage to need another command.  It means
the user has to decide up front which one she needs.  Showing the rest
of matches after the exact one doesn't have this disadvantage.

Once again, this bug is about the UI.  The UI shows only one match
with one back-end, and all of them with another.  That's the problem I
think we should address.

>     I guess the elisp-mode back-end returns just one candidate, whereas
>     the etags back-end returns a list.  But it's confusing to have such
>     evident differences just because you changed the back-end, I think.
> 
> etags showing all matches is, in general, the limitation of the tags file format.

??? How can what we show be the limitation of the format of the
database where we keep the data?  If we think we should not display
everything, let's filter the irrelevant stuff before showing it.

> But if someone submits a patch making it more precise without creating false negatives, I'm all for it.

I'd generally expect the people who worked on a feature and lobbied
for its introduction to be the ones who work on such significant
issues with that feature.  But if not, I hope that someone
materializes soon.

> And hey, you changed the back-end. It uses a different mechanism under the covers. *Of course* you're going to get different results, in many cases.

I changed the back-end, not the UI.  The UI should not depend on the
back-end so much -- this is what this thread is all about.  I'm okay
with getting slightly different results, and I'm okay with having user
options to control the amount of filtering so that I could get
significantly more or less results when I need that.

But presenting such widely different results _by_default_ is a
terrible usability problem.  Just imagine a user who needs to switch
languages.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 01:15:16 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>     I certainly hope that at least the Semantic one materializes soon
>     enough, otherwise it sounds like all this move to xref was for the
>     benefit of unbundled packages, and users of Emacs are just punished by
>     having to learn a new UI for no real advantage.
> 
> It's probably up to CEDET developers.

I hope they are reading this.  (Personally, I'd expect the back-ends
for at least the most popular languages be ready before the switch to
the new API and the new UI, but that's me.)

> But the UI itself is one of the benefits. If the old one was considered superior, the new API could have been introduced without really changing the UI.

I didn't ask for the old UI; I can still have it if I want it.  This
bug report is about improving the new UI.

>                 . The doc string of xref-find-function mentions several variants of
>                   invoking the function, but there doesn't seem to be any way of
>                   controlling that when invoking the function interactively, is
>                   there?  I think it would be good to be able to lookup only the
>                   definitions or only the references of a symbol.
> 
> There seems some confusion here: you do now invoke xref-find-function. It's not a function anyway, it's a variable, which different backends can set to implement the backend interface.

Then the confusion is the doc string's fault: it should make this
aspect clear.  And if there are no back-ends currently that support
these options, I'd suggest to remove that from the doc string -- we
shouldn't advertise in the docs stuff that no one implemented.

>     Not from me: I've been told in this discussion that I don't understand
>     the new UI, don't appreciate its many virtues, and generally am no
>     more than an obstacle on the way of progress.
> 
> I really don't remember stating that in this discussion.

Indeed, you didn't.  But you are not alone in this discussion.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 01:25:14 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>     The different ways to call xref-find-function are to distinguish
>     jumping to the definition or to "all uses", and currently few backends
>     support those features, I don't think we have a clear idea yet of how
>     they should be presented to the user, and there are too many variants to
>     collapse them all into C-u.  So they'd probably be provided via
>     different commands instead of all being accessed via M-.
> 
> M-x xref-find-references, for starters.

It doesn't seem to work for me: it says "no references found" whatever
symbol I try, even those I know are referenced.  What am I missing?

> - If I do have a tags table loaded, it's likely for a different project than Emacs. This is most likely true for the majority of our users.

etags.el supports more than one loaded tag table, so this shouldn't be
a problem.

> I think Eli (and everyone like-minded) should advise elisp-xref-find and xref-identifier-completion-table-function instead, inside their init script, like suggested by Daniel in another thread.

Please don't make advice the default Emacs customization method.  User
options are much easier to use, so let's do what's possible with them,
and leave advice etc. for more exotic use cases.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 01:32:35 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>     We shouldn't treat the Emacs core as second-rate citizen, either.  A
>     feature as central to code development and Emacs in general as finding
>     definitions and references of symbols cannot thrive only (or mainly)
>     outside the core, IMO.
> 
> So? What are you doing to help with that?

I filed this bug report, for starters.  I then invested a non-trivial
amount of my time into learning this feature, trying different things,
describing my experiences, thinking about ways to improve this, etc.
And I continue doing that.  I hope this does help; if not, feel free
to close this bug report as "won't fix" and tell me not to bother
anymore.

> Are you asking anyone currently participating in this thread to work on CEDET?

That would be nice, yes.  But I'd settle with merely having us agreed
to the above statement.

>     Where did you see me say I dislike it?  I'm just saying that learning
>     a new UI for the sake of a new UI is a waste.  I _am_ prepared to
>     learn a new UI if it brings new useful functionality with it.
> 
> The new UI itself brings tangible improvements by itself, as well as a unified interface that can (and hopefully will) be used by different projects. I don't know how you don't see the value in that.

I'm saying that improvement that is only a potential one, without any
feature that actually uses it, is incomplete, to say the least.  If
there are no back-ends that make use of these niceties, they simply
don't exist as far as users are concerned, and consequently their
value is purely theoretical.

This feature, including the new UI and API, was introduced 4 months
ago, and we still have only 2 back-ends supporting it: ELisp and Ada.
All the other languages, including C/C++/Java, and other important
ones, still use etags, which is said to be inadequate.  Is someone
working on more back-ends as we speak?  If not, I see currently no
practical benefits from introducing these changes.  They are nothing
more than infrastructure waiting for programmers to use it, and those
programmers don't seem to be coming.  Putting on my cynic hat, I'd say
it's a solution looking for the problem.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 01:44:15 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>         btw the defun I'm probably after is not the first or second hit there;
>         looks like the candidates are arranged in the alphabetical order of
>         the *.el files).
> 
>     Indeed, the ordering looks incorrect.  This is a bug.
> 
> The idea here is to group candidates belonging to different files together. Sorting the files alphabetically after that is the first natural choice, but how else to do it better?

You could have the few "best matches" listed separately in the
beginning of the list, followed by the grouping shown today.

> - Should xref always try to preserve the order in which xrefs are returned?

No!  This would make the UI even more dependent on the back-ends.

> - Suppose one xref in a.el is returned before all xrefs in b.el. And another xref in a.el is returned after. In what order will we display a.el and b.el?

See above: the best matches could be listed irrespective of the files
they were found in.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 01:45:48 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>     Maybe we should write an xref-old-ui-mode.
>     It would make M-. always prompt and it would always jump to the first
>     entry and then let you use C-u M-. to jump to the next match.
> 
> That's doable, but it's not high on my priorities list. In case anyone's wondering.

I don't think anyone asked for it, so feel free not to do anything
about that.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Eli Zaretskii <eliz@gnu.org>, 19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 18:54:05 -0400
> 
> > - If I do have a tags table loaded, it's likely for a different project than
> >   Emacs.  This is most likely true for the majority of our users.
> 
> I really think the etags backend should only return data when the TAGS
> file is in one of the parents of the current directory.

That goes back to the "working on several projects" discussion of
yore.  Specifically, you might be working on two related packages
(e.g., one calls the other) that live in two different directories.

> And it should also be able to keep several independent TAGS tables
> opened for different projects in different directory trees.

It already does that.

> > I think Eli (and everyone like-minded) should advise elisp-xref-find and
> > xref-identifier-completion-table-function instead, inside their init script,
> > like suggested by Daniel in another thread.
> 
> It seems a common enough request that we should accommodate
> them better. At the very least provide a minor mode which sets up
> such advice.

User options if possible, minor modes if not, yes.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 02:15:12 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: 19468@debbugs.gnu.org
> 
>     It seems a common enough request that we should accommodate
>     them better. At the very least provide a minor mode which sets up
>     such advice.
> 
> But how common is it, really? I've only seen one person ask for it so far. Surely the 5 or so people who actually use tags with Elisp are a distinct minority compared to the whole Emacs user base.

Why are you only talking about Elisp?  What about other languages?
Emacs exists not only for developing Emacs or Emacs-related packages.

More importantly, this bug report is not about my preferences
(although I did describe them when asked).  It is about the
inconsistencies and incompleteness of the new UI.  Let's talk about
that, and not about personal preferences.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Eli Zaretskii <eliz@gnu.org>, 19468@debbugs.gnu.org
> Date: Mon, 27 Apr 2015 19:19:16 -0400
> 
> Can't etags do better than substring matching and only return matches
> that actually have the requested name (and maybe fallback on substring
> matching if there's no exact match)?

See the various *-match-p functions in etags.el, which can be used for
that.  You will see in etags-recognize-tags-table that etags.el
programs the order in which the matches are found into the
find-tag-tag-order variable.  We could remove some of the criteria
from that list, preferably under control of some user option, thus
making the results more accurate.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 02:47:41 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>           emacs -Q
>           M-x xref-et TAB => [No match]
> 
>           It should be autoloaded, IMO.  Perhaps there should also be a
>           global minor mode, for those who'd like that.
> 
> I don't know if we want to settle on that solution, actually. Have you seen this thread?
> 
> http://lists.gnu.org/archive/html/emacs-devel/2015-04/msg01236.html

Yes, but I'm unsure how is that relevant.  Are you talking about
auto-loading, or are you talking about a global minor mode?

> That's not so easy to wrap in a global minor mode. I'm not even sure a minor mode would be a good approach here at all.

Sorry, you lost me.  What aspects prevent us from making a global
minor mode that uses xref-etags-mode in all buffers?

>         . It does nothing when point is on a unique symbol:
> 
>           M-. set_cursor_from_row RET
>           Now, with point on its position under set_cursor_from_row, type
>            M-. again => nothing(??!!) happens
> 
>           I guess it looks for more symbols matching set_cursor_from_row,
>           finds out that this is the only one, and does nothing.  This is
>           not useful.  It should at the very least say something like
>           "set_cursor_from_row: this is the only match".
> 
> `M-x find-tag' behaves exactly the same: even when the result is that we don't move anywhere (for the same reason), there's no helpful message. Just "Mark set" in the echo area.

How is the old UI relevant here?  We are talking about the new UI.  (I
already explained elsewhere that the old UI had "C-u M-." to go to the
next match, whereas the new UI provides the *xref* buffer for that
instead.  So what the old UI did in this situation is not relevant,
because the crucial feature of going to the next match, which was part
of dealing with this situation in the old UI, is missing in the new
UI.  Thus, the new UI should find a different solution for this
situation.)

> I don't know if adding a new message in this case will be too helpful, but you're welcome to suggest the wording.

I thought I already did, see above.

>         . Some problems with key bindings in the *xref* buffer:
> 
>           . RET displays the candidate listed on the current line, but
>             closes the window displaying *xref*, so it's not easy to try
>             another candidate afterwards.
> 
> There's also the `C-o' binding, which displays the xref, but keeps the current focus.

OK (did I say the documentation is missing?), but RET is the usual way
of doing this, so I guess many users will.

> >                                        I think it would be more helpful
> >         to just switch to the window showing the definition, but leave
> >         the *xref* buffer shown.
> 
> Some packages take this approach; I don't like it. But this should be easy to implement with a user option that would slightly change the behavior of `xref-goto-xref'. Care to take a stab at it?

Maybe, if/when I have time.  The more important issue is how to go to
the next candidate once the window showing *xref* is closed.

>           . You need to use the unconventional '.' and ',' to both move and
>             display the definitions -- this should at least be mentioned in
>             the initial echo-area message when *xref* is first displayed.
>             (This was reported as a problem in the original report, but
>             seems to be left unchanged.)
> 
> Should those be `n' and `p' instead, by default? I've found myself using these bindings very rarely anyway, and `n' is still very close.

Possibly.

>          (Other similar Emacs modes,
>           like Compilation and Grep, do provide commands to move to the
>           next item without first switching to the buffer that shows all
>           the hits.)
> 
> Compilation and Grep both use M-g M-n/M-g M-p, and the next-error-function variable. Do you want xref to reuse it?

Something like that, yes.  There's also "C-x `".  Or maybe you should
bring the equivalent of tags-loop-continue back ;-)

> Would you be comfortable with forgetting the current list of errors after navigating somewhere with xref?

No, I don't think so.

>           But what are the alternatives, if any?  I could only find
>           something related in ada-mode and in elisp-mode.  This means
>           that, for example, for C/C++ and Java, etags is the only
>           available back-end, and this change is currently just a different
>           UI wrapping the same basic functionality?  Is there any further
>           development planned for the near future?
>
> There's also ggtags in GNU ELPA. I'm sure it could provide some integration, too.

I don't see any xref-related code in ggtags.  Did I miss something?

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 03:24:03 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: 19468@debbugs.gnu.org
> 
>     That makes a lot of sense when the candidates are equally likely to be
>     The One that the user's looking for.  In the case of etags I think the
>     problem comes from the substring matching it uses.
> 
> etags puts some effort into returning the candidates in the most plausible way. Should we ignore that? etags--xref-find-definitions tries to honor it, at least according to the comment.

etags--xref-find-definitions doesn't honor the etags order well
enough, because it groups the matches by file, whereas the etags
back-end returns them ungrouped.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Tue, 28 Apr 2015 05:15:04 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: 19468@debbugs.gnu.org
> 
>     Could be.  But etags has been around for a long time and was the only
>     "standard feature" of the kind until very recently in Emacs, so I think
>     it's fair to assume that there are more such users out there.
> 
> etags might be popular (so some sort a general solution might be welcome), but that doesn't mean it's used with Elisp a lot.

What alternatives were there, before xref?

> For instance, if we consider Debian-like distributions, https://packages.debian.org/jessie/all/emacs24-el/filelist does not include the TAGS file. I don't think the C sources are generally included in the binary distributions.

It's easy to generate TAGS for Lisp files, if one needs them.  We
describe all that in the user manual.

> And if we're just talking about Emacs developers, I wonder why nobody else spoke up on emacs-devel or in this bug yet.

This goes both ways: no one spoke about xref, either.  I wouldn't
assign any significance to that.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 06:01 PM, Eli Zaretskii wrote:

> etags--xref-find-definitions doesn't honor the etags order well
> enough, because it groups the matches by file, whereas the etags
> back-end returns them ungrouped.

That's incorrect. That function doesn't group anything.

The xref UI does the grouping.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:48 PM, Eli Zaretskii wrote:

> By
> contrast, the new UI is inconsistent: with some back-ends it shows the
> list of matches and allows to navigate it, with others it shows only
> the first match and does not give any way to get the next match.

The UI has nothing to do with it. It's just the etags backend often 
returns multiple results to the "find-definitions" command.

The elisp backend is more precise, but it can also return multiple 
results under certain conditions. For instance, when the exact name 
we're looking for corresponds to a function, a feature name, and a face 
(there's certain effort there not to show both a function and a variable 
with the same name, because they usually both point to a minor mode 
definition).

>> No: there's only one definition of `find-tag' so Elisp's xref backend
>> just returns that one and that's it.
>
> There's more than one matching definition, see below.

If you want "matching" definitions, use xref-find-apropos.

> But that's largely immaterial: this bug report is not about the
> back-end, it's about the UI.  The UI should be independent of the
> back-end, otherwise the users will be confused when they switch
> between languages.

The UI can only work with that a backend returns to it. A non-ideal 
implementation can result in non-ideal behavior in the end.

> (If it turns out that some back-ends need to be
> augmented so that they allow the front-end to present similar UI for
> the same query, then those back-ends should be enhanced.)

Sure. I'd be happy to leave that to someone else, but there doesn't seem 
to be someone actively maintaining it.

In that situation, we first try to make the backend work as well as 
`find-tag'. Patches, as well as specific technical suggestions, are very 
welcome.

>> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>> if org-ctags.el happens to be loaded
>
> ??? Why should the xref matches depend on what is and isn't loaded?

The above is referring to the Elisp xref backend, which is based on the 
package `find-func'. Which, yes,

> That would make xref exactly equivalent to "M-x apropos", which means
> this mode of operation would make xref entirely redundant.

I don't know how to respond to this. Yes, they use the same data 
available at runtime. No, they're not exact duplicates of each other; 
the data is used in slightly different ways and presented in a different 
interface.

> To say
> nothing of the fact that this doesn't scale to any language except
> ELisp.

Yes, the Elisp backend doesn't scale to other languages.

> (One of my worst annoyances is to type a
> C-h command and be presented with "[No match]", because some package
> is not loaded or some function is not available in the Emacs
> configuration I happen to be using.)

It's the cost of doing business, as far as I'm concerned. You can't use 
tags for non-core Elisp code anyway, such as anything in your init 
directory (installed packages, etc), and any Elisp files installed 
separately by the operating system's distribution.

Actually, if you're not working on a Git checkout, I don't think you can 
use the tags even for the Elisp code that's part of Emacs.

> The important point for the purposes of this bug report is that the
> stuff presented by the UI for the same query does not differ so
> radically when you switch the back-end.

In general, it's the concern of each particular backend, to adhere to 
the described interface. The UI has nothing to do with it.

>    . there should be a way to go to the next/previous match if the
>      *xref* buffer is not displayed (or not created in the first
>      place); if this happens because there's only one match, we should
>      say so explicitly

I don't see the point. That fact that we've jumped to a new location 
means there was a match, and since we haven't seen the xref buffer, 
there were no other matches.

The message might feel assuring to you in the beginning, but over time I 
think it'll just become a nuisance.

>    . when there are more than one possible match, the UI should behave
>      similarly, i.e. display the *xref* buffer; when there's only one
>      match, it should _never_ display *xref*,

That's how it works already. With the exception of the "possible match" 
phrasing, which is nonsense to me. There either are matches of the kind 
that the current command is asking for, or there aren't.

>    . if the criteria for filtering of the matches are very different
>      between the possible back-ends, there should be some agreed-upon
>      uniform default that returns roughly the same number of matches
>      with all back-ends, and the rest should be controlled by user
>      options

These criteria are described in the xref-find-function docstring. Maybe 
not ideally: feel free to suggest a more clear wording.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 04:30 PM, Stefan Monnier wrote:

> The reason is because that `completion-metadata' says that the
> completion category is `buffer' and because completion-category-defaults
> says that for `buffer' completion, we should use the `substring' style
> by default.

Hmm, maybe we should introduce a new category, for reading symbols. 
Aside from xref, we could use it in describe-function/variable/etc.

Although the fact that it tries prefix matches first, and only falls 
back to substring matches, doesn't look too terrible to me.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:55 PM, Eli Zaretskii wrote:

> You could have the few "best matches" listed separately in the
> beginning of the list, followed by the grouping shown today.

What's "best matches"? I suggested using `tag-exact-match-p' in the 
etags backend. What's your take on this?

>> - Should xref always try to preserve the order in which xrefs are returned?
>
> No!  This would make the UI even more dependent on the back-ends.

Without using the order, in which matches are returned by the backend, 
we can't even know what the "best matches" are.

What about the rest of the predicates in `find-tag-tag-order' set by 
`etags-recognize-tags-table'? Should we disregard all of them?

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:49 PM, Eli Zaretskii wrote:

> Some would say it's a disadvantage to need another command.  It means
> the user has to decide up front which one she needs.  Showing the rest
> of matches after the exact one doesn't have this disadvantage.

On the plus side, if there's just one exact match, Emacs will jump to it 
automatically.

> Once again, this bug is about the UI.  The UI shows only one match
> with one back-end, and all of them with another.  That's the problem I
> think we should address.

UI shows all matches that a backend returns.

> ??? How can what we show be the limitation of the format of the
> database where we keep the data?  If we think we should not display
> everything, let's filter the irrelevant stuff before showing it.

It would be a limitation with the format if any `tag-exact-match-p' 
implementation can't be 100% reliable. Some prior experience dealing 
with it told me that may be the case.

But anyway, I've just opened lisp/TAGS, and I haven't been able to make 
`(tag-exact-match-p "...")' return t with any input and position I've tried.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:55 PM, Eli Zaretskii wrote:

> I filed this bug report, for starters.  I then invested a non-trivial
> amount of my time into learning this feature, trying different things,
> describing my experiences, thinking about ways to improve this, etc.
> And I continue doing that.  I hope this does help; if not, feel free
> to close this bug report as "won't fix" and tell me not to bother
> anymore.

Let's concentrate on the interface and the current implementations instead.

> This feature, including the new UI and API, was introduced 4 months
> ago, and we still have only 2 back-ends supporting it: ELisp and Ada.

And etags. Three backends isn't exactly the same as "no backends".

> They are nothing
> more than infrastructure waiting for programmers to use it, and those
> programmers don't seem to be coming.  Putting on my cynic hat, I'd say
> it's a solution looking for the problem.

I didn't see programmers running to improve the built-in C support in 
Emacs before. Certainly nobody expected to see them running now, simply 
because xref was introduced.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 06:03 PM, Eli Zaretskii wrote:

> What alternatives were there, before xref?

1. M-x find-function (some even used M-. as its binding).

2. C-h comment-forward RET, then click on the file name.

> It's easy to generate TAGS for Lisp files, if one needs them.  We
> describe all that in the user manual.

Could you point me to the exact section?

> This goes both ways: no one spoke about xref, either.  I wouldn't
> assign any significance to that.

Judging by a few comments by several people, it's being used.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:58 PM, Eli Zaretskii wrote:

> Why are you only talking about Elisp?  What about other languages?
> Emacs exists not only for developing Emacs or Emacs-related packages.

Simply because this subthread of discussion, for me, is about using 
etags in emacs-lisp-mode.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:50 PM, Eli Zaretskii wrote:

> I hope they are reading this.  (Personally, I'd expect the back-ends
> for at least the most popular languages be ready before the switch to
> the new API and the new UI, but that's me.)

CEDET has been lagging quite a bit. Even the 
completion-at-point-functions (available since 24.1) support was only 
added half a year ago.

>> There seems some confusion here: you do now invoke xref-find-function. It's not a function anyway, it's a variable, which different backends can set to implement the backend interface.
>
> Then the confusion is the doc string's fault: it should make this
> aspect clear.  And if there are no back-ends currently that support
> these options, I'd suggest to remove that from the doc string -- we
> shouldn't advertise in the docs stuff that no one implemented.

The doc string is about the interface: as long as a given backend 
implements the `references' action, `M-x xref-find-references' will 
display the returned list.

Please feel free to suggest better wording for the docstring.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 05:52 PM, Eli Zaretskii wrote:

>> M-x xref-find-references, for starters.
>
> It doesn't seem to work for me: it says "no references found" whatever
> symbol I try, even those I know are referenced.  What am I missing?

Do you know how to implement the "all uses" search using only a tags 
file? I don't. Likewise for Elisp.

So, sorry to say, both currently implemented backends don't support it. 
ggtags, in GNU ELPA, might.

> etags.el supports more than one loaded tag table, so this shouldn't be
> a problem.

If the only table I'm using is of my current Ruby project, that would be 
a problem anyway. Still, Stefan's suggestion of looking up the directory 
tree should solve that.

> Please don't make advice the default Emacs customization method.  User
> options are much easier to use, so let's do what's possible with them,
> and leave advice etc. for more exotic use cases.

I'm suggesting its use lightly. As you've noticed, a minor mode was the 
first solution. However, if people want to use the Elisp xref backend is 
many other situations, you'll run out of hooks to add xref-etags-mode to.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 06:00 PM, Eli Zaretskii wrote:

> Yes, but I'm unsure how is that relevant.  Are you talking about
> auto-loading, or are you talking about a global minor mode?

About replacing xref-etags-mode entirely, and using advice to use tags 
wherever elisp-xref-find is used.

>> That's not so easy to wrap in a global minor mode. I'm not even sure a minor mode would be a good approach here at all.
>
> Sorry, you lost me.  What aspects prevent us from making a global
> minor mode that uses xref-etags-mode in all buffers?

Um, yeah, it would work, if you want xref-etags-mode in *all* buffers at 
all, including those that might use yet another different backend.

Actually, it still looks a good option for you, AFAICT.

> How is the old UI relevant here?  We are talking about the new UI.

We need something to compare to. If I compare it to a popular IDE, they 
usually have this functionality bound to Ctrl-Click. If there's just one 
match, the user is taken to its location, if there are several, a 
context menu is shown. No message about the only match.

Which makes sense to me.

> (I already explained elsewhere that the old UI had "C-u M-." to go to the
> next match, whereas the new UI provides the *xref* buffer for that
> instead.  So what the old UI did in this situation is not relevant,
> because the crucial feature of going to the next match, which was part
> of dealing with this situation in the old UI, is missing in the new
> UI.  Thus, the new UI should find a different solution for this
> situation.)

Where in the old UI you could go to the next match, in the old UI we 
display the xref buffer with all matches. Where this buffer is not 
displayed, in the old UI you wouldn't have been able to the next match.

> I thought I already did, see above.

Okay, sorry. It doesn't sounds like a good idea to me anyway. Especially 
when, in many environments (like Elisp), finding only one match will be 
the norm, not exception.

> OK (did I say the documentation is missing?), but RET is the usual way
> of doing this, so I guess many users will.

They could also use winner-mode, or switch to *xref* manually.

Where in the documentation would you mention the key bindings?

>> Should those be `n' and `p' instead, by default? I've found myself using these bindings very rarely anyway, and `n' is still very close.
>
> Possibly.

I'll wait for a stronger "yes" on this.

> There's also "C-x `".  Or maybe you should
> bring the equivalent of tags-loop-continue back ;-)

We've already taken up its binding, and since showing the whole list is 
a replacement for the "show first, allow to iterate through the rest" 
logic, we're unlikely to use a new binding for the `tags-loop-continue' 
equivalent.

The next-error commands might make sense for what you're asking, as long 
as we've all considered the interaction with other commands that set 
next-error-function.

>> Would you be comfortable with forgetting the current list of errors after navigating somewhere with xref?
>
> No, I don't think so.

But that what happens if you run a compilation (with errors), and then 
call Grep. Doesn't it?

> I don't see any xref-related code in ggtags.  Did I miss something?

Nope. It remains to be written.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/28/2015 04:36 PM, Stefan Monnier wrote:

> Indeed, and it's largely unrelated since it would be a patch to the
> xref/etags backend and wouldn't touch elisp-mode.el.

On the other hand, tighter integration might have its benefits, too. 
Otherwise, the user wouldn't be able to use the Emacs's TAGS files for 
navigation when working on, say, an ELPA checkout.

I'm not sure if somebody will really want to, but it makes sense to be 
able to use it both from core Elisp code, and third-party one equally.

The tags part of the emacs-lisp-mode's xref backend could bind etags 
vars locally based on the value of source-directory, instead of using 
the current tags-table-list.

> As mentioned, this suggestion is not directly related to other backends,
> so whether xref/elisp completely shadows xref/etags or not is orthogonal.
> BTW: Do you have a lisp/TAGS file?  If so, why?

That's a good question. Indeed, we could say I only had it built to see 
how tags work with Elisp.

However, I do occasionally use src/TAGS, and 'make tags' builds both files.

Re: UI inconveniences with M-.

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
>> > and if you type the same one, you get nowhere.  The other matches are
>> > only available via completion, see below.
>> Maybe we should write an xref-old-ui-mode.
> I didn't ask for the old UI.  (If I want it, I can still have it,
> since the etags.el commands are still available.)  I'm okay with the
> new UI.  This bug report is about the new UI, and its goal is to make
> the new UI better.

Then I don't understand what you mean by the text I quoted.  E.g.:

   if you try "C-u M-.", you get prompted for the symbol again,
   and if you type the same one, you get nowhere

Well, duh!  What did you expect?  In the new UI, "C-u M-." jumps to
*the* definition (the "C-u" is only used to let you type the name of
thing you're looking for), so of course if you just jumped to it, asking
to jump to it again will get you nowhere else.  I guess it would be OK
to make M-. emit a message like "Hello?  We're already here!" in this
corner case, if you want to make sure something happens.

> old UI was consistent and complete: it always displayed the first/next
> match and provided a command to go to the next/previous one.  By
> contrast, the new UI is inconsistent: with some back-ends it shows the
> list of matches and allows to navigate it, with others it shows only
> the first match and does not give any way to get the next match.

In the "M-. find-tag" example, the UI does let you see all matches.
It's just that there's only one.  If you want, we could make this case
popup an *xref* buffer with a single entry, but I think it makes more
sense to just jump straight to that one entry instead.

> But that's largely immaterial: this bug report is not about the
> back-end, it's about the UI.

I wouldn't be so sure.

>> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>> if org-ctags.el happens to be loaded
> ??? Why should the xref matches depend on what is and isn't loaded?

Because that's how xref/elisp works.

> That would make xref exactly equivalent to "M-x apropos", which means
> this mode of operation would make xref entirely redundant.

To you, maybe.  Personally, I find it's *much* quicker to use M-. than
to use C-h f RET <go-to-the-link> RET, and it has the added benefit that
M-, brings me right back if I need to.

> To say nothing of the fact that this doesn't scale to any language
> except ELisp.

Not at all.  Many (most?) packages which offer a functionality along the
lines of "M-. to jump to the definition" use an implementation technique
which is fundamentally similar/equivalent (obviously, they don't query
a running Emacs, but they query a running REPL).

> We could offer this mode as an optional feature, but it certainly
> shouldn't be the default.

Many users here disagree.

> (One of my worst annoyances is to type a C-h command and be presented
> with "[No match]", because some package is not loaded or some function
> is not available in the Emacs configuration I happen to be using.)

Yes, that's a long standing problem, indeed.  I have a local hack which
tries to add to loaddefs.el a concise description of where definitions
might be found (basically, for each file, list the few prefixes used in
that file), so that "C-h f" can load those packages when needed.
It's "proof of concept" and it has occasionally rendered my Emacs
session unusable (e.g. because it ended up deciding to try and load,
say, w32-fns.el to see if it might contain the definition).  I think
I've now solved most of those problems in the w32-* files, but the
underlying risk is still there.

> That's one way of looking at it.  Another is to say that etags gave
> you more information and thus allowed to find more loose matches,
> which is helpful when your memory is failing you.

M-. jumps to the definition of the "thing under point".  There's no
memory involved.
If you're not sure what you're looking for, then you're expected to use
the completion facilities in C-u M-.

> But in the context of this bug report, that, too, is immaterial: if we
> think the etags back-end gives too much information, by all means
> let's filter it before presenting the matches.

Yes, that's clearly very much needed.

> Bonus points for making the filtering optional, since some people
> might like that, and in some situations even you might need it.

I disagree.  The filtering is needed by the design of the xref API.
If you want to see the "unfiltered" data, then use M-x xref-find-apropos.

> Then let's fix the results we display with the etags back-end to show
> only the relevant ones.

Yes, please.

> Saying that the back-end returns a confusingly large amount of bogus
> matches, and then switching to a UI that assumes much smarter
> back-ends (which don't really exist) makes very little sense to me.

The smarter backends already exist, in so far as there is already code out
there which provides the needed functionality.  It just doesn't use the
xref API yet, because it was written before xref.el.

> The old code attempted to support both use cases, by showing the exact
> match(es) first, followed by less likely ones.  We do similar things
> in other commands.  The advantages of such a method are that (1) you
> don't need to second-guess the user, (2) you provide only one command
> for the user to remember, and (3) if the user only _thinks_ she knows
> the name, but really doesn't, she can find the information without
> having to invoke another command.

>> I think that's an important feature of the new code in this respect.

> But it is incomplete without a means to get to the other possible
> matches in this case.

> To summarize my points in this sub-thread:

>   . the UI should depend much less on the back-end, ideally not at all

You keep repeating this, but that is absolutely meaningless to me (kind
of like "it doesn't work" in bug reports):
- This consistency was trivially obtained with the old etags.el code
  since it had one 1 backend.
- What the fuck should the new UI do if one backend returns 1 match and
  another returns a hundred (as in your "find-tag" example)?

>   . there should be a way to go to the next/previous match if the
>     *xref* buffer is not displayed (or not created in the first
>     place); if this happens because there's only one match, we should
>     say so explicitly

The only case where the *xref* is not displayed is when there's only
1 match returned by the backend.  So "go to the next/previous match" is
again meaningless.

>   . when there are more than one possible match, the UI should behave
>     similarly, i.e. display the *xref* buffer; when there's only one
>     match, it should _never_ display *xref*,

That's exactly what the code does.

>     and should display an indication of the fact that there's only one
>     match

The current code "displays" this indication by jumping to the sole match
instead of jumping to the *xref* buffer.  I think it's clear enough.

>   . if the criteria for filtering of the matches are very different
>     between the possible back-ends, there should be some agreed-upon
>     uniform default that returns roughly the same number of matches
>     with all back-ends, and the rest should be controlled by user
>     options

I don't see what that could concretely look like.


        Stefan

Etags.el (was: UI inconveniences with M-.)

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

>> I really think the etags backend should only return data when the TAGS
>> file is in one of the parents of the current directory.
> That goes back to the "working on several projects" discussion of
> yore.  Specifically, you might be working on two related packages
> (e.g., one calls the other) that live in two different directories.

And as mentioned back then, this would be solved by providing a way for
the user to tell etags.el about the link between those two projects.
For the case where this link only exists during the running session, we
could simply use the existing "visit-tags-table" functionality where you
can add a TAGS file to the one(s) already loaded, so the new
functionality would behave similarly to the old one.

>> And it should also be able to keep several independent TAGS tables
>> opened for different projects in different directory trees.
> It already does that.

No it doesn't.  It lets you use the union of a set of TAGS tables, which
means that the TAGS tables of independent projects pollute each other.


        Stefan

Re: UI inconveniences with M-.

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

> Hmm, maybe we should introduce a new category, for reading symbols.  Aside
> from xref, we could use it in describe-function/variable/etc.

Sounds fine to me.  Tho "symbol" might be a tad too generic.

> Although the fact that it tries prefix matches first, and only falls back to
> substring matches, doesn't look too terrible to me.

Not sure what you're saying here.  Indeed, for buffers,
completion-category-defaults says to first try prefix matching and
fallback on substring only if that fails.  This was done mostly to
reduce friction when the change was introduced.


        Stefan

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:13 AM, Stefan Monnier wrote:

> Sounds fine to me.  Tho "symbol" might be a tad too generic.

...identifier? Better name welcome.

> Not sure what you're saying here.  Indeed, for buffers,
> completion-category-defaults says to first try prefix matching and
> fallback on substring only if that fails.  This was done mostly to
> reduce friction when the change was introduced.

I was saying that about the current identifier completion in `C-u M-.'. 
And was mistaken: it's close, but uses partial-completion (according to 
`completion-styles'). Which is close, but a bit less user-friendly.

Re: UI inconveniences with M-.

[Stefan Monnier]

>> Sounds fine to me.  Tho "symbol" might be a tad too generic.
> ...identifier? Better name welcome.

I originally suggested "identifier" and then changed my mind before
sending the message: the problem, really, is that things like M-x
completion also complete identifiers/symbols yet I'm not convinced they
should use the same category.
Then again, maybe they should.

> I was saying that about the current identifier completion in `C-u M-.'. And
> was mistaken: it's close, but uses partial-completion (according to
> `completion-styles'). Which is close, but a bit less user-friendly.

Of course, a better option might be a new style `pc-substring', which
combines partial completion and substring, so that "f-t" matches not
only things like "find-tag" but also "buffer-face-toggle".

That's been on the "todo" for "ever", but I just got used to typing
"*f-t" or "-f-t" when I need this kind of functionality.


        Stefan

Re: UI inconveniences with M-.

[Helmut Eller]

On Tue, Apr 28 2015, Stefan Monnier wrote:

> [ Moving this to emacs-devel.  ]
>
>>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
>>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
>>> > and if you type the same one, you get nowhere.  The other matches are
>>> > only available via completion, see below.

I my proposal the next-error/prev-error commands could be used to
navigate through xrefs even after the *xref* buffer disappeared.  This
can was done--quite elegantly I think--by setting next-error-last-buffer
to the *xref* buffer and set the buffer local value of
next-error-function in the *xref* buffer appropriately.

Dmitry took that out because he thinks that it interferes too much with
compilation-mode.  Maybe he's right but let me note that M-x rgrep also
uses next-error/prev-error and similarly interferes with
compilation-mode.

Helmut

conflicting uses of next-error-function (was: UI inconveniences with M-.)

[Stefan Monnier]

> Dmitry took that out because he thinks that it interferes too much with
> compilation-mode.  Maybe he's right but let me note that M-x rgrep also
> uses next-error/prev-error and similarly interferes with
> compilation-mode.

Indeed, I've occasionally been annoyed by conflicting uses of
next-error-last-buffer/next-error-function.  And in both directions:
sometimes I don't want `grep' to hijack my ongoing traversal of
compiler warnings, and sometimes I regret that some other package
doesn't let me use C-x `

Maybe we should consider refining this so that all package which can
sometimes benefit from C-x ` would feel free to provide support for
next-error, without the fear of imposing themselves.  For that, the
next-error framework needs to make it possible for the user to control
which package gets to really use next-error and when.

Not sure what that should look like, tho.  Maybe some config var could
specify which packages's next-error-support is "enabled" by default,
then some way for the user to override that default on a case by case
basis: e.g. a command to run inside the *xref* buffer would
"force-enable" the next-error support for this one time, and maybe
a new `next-error-pop-context' command would go back to the previous
"next-error-last-buffer/function"?


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Vitalie Spinu]

 >>> Stefan Monnier on Mon, 27 Apr 2015 18:54:05 -0400 wrote:

 >> - If I do have a tags table loaded, it's likely for a different
 >> project than Emacs.  This is most likely true for the majority of
 >> our users.

 > I really think the etags backend should only return data when the
 > TAGS file is in one of the parents of the current directory.

 > And it should also be able to keep several independent TAGS tables
 > opened for different projects in different directory trees.

This is close to perfect IMO. In addition I would like to be able to
"link" tag tables.

Let's say I have two projects A and B. By "linked" tag tables I mean
that both tag tables are simultaneously accessible from descendant files
of path/to/A and path/to/B.

The UI is essentially one command - `etag-link-tag-table`. If I call
`etag-link-tag-table` from project A and choose a tag table in B then A
and B tag tables become linked, and vice versa. Simple as that.


  Vitalie

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:00 PM, Vitalie Spinu wrote:

> Let's say I have two projects A and B. By "linked" tag tables I mean
> that both tag tables are simultaneously accessible from descendant files
> of path/to/A and path/to/B.

The etags format already has a similar feature (etags --help | grep 
include), but it's only one-way, AFAIK.

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Helmut Eller on Wed, 29 Apr 2015 08:10:45 +0200 wrote:

 > On Tue, Apr 28 2015, Stefan Monnier wrote:
 >> [ Moving this to emacs-devel.  ]
 >> 
 >>>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
 >>>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
 >>>> > and if you type the same one, you get nowhere.  The other matches are
 >>>> > only available via completion, see below.

 > I my proposal the next-error/prev-error commands could be used to
 > navigate through xrefs even after the *xref* buffer disappeared.  This
 > can was done--quite elegantly I think--by setting next-error-last-buffer
 > to the *xref* buffer and set the buffer local value of
 > next-error-function in the *xref* buffer appropriately.

 > Dmitry took that out because he thinks that it interferes too much with
 > compilation-mode.  

How does it interfere with the compilation-mode more concretely?

By definition next-error(M-g n)/previous-error(M-g p) operate on the
most recent buffer (in buffer-list) that contains `next-error-function`
and is complete unrelated to compilation mode. If you just called
`occur` you navigate in *occur* buffer, grep - in *grep*, compile - in
*compile* etc.

IMW *xref* is conceptually closer to *compile* than to *occur* or even
*grep* buffers.

Dmitry, would you mind putting this back?

Thanks,

  Vitalie

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 00:00:31 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>         No: there's only one definition of `find-tag' so Elisp's xref backend
>         just returns that one and that's it.
> 
>     There's more than one matching definition, see below.
> 
> If you want "matching" definitions, use xref-find-apropos.

That's a UI inconvenience, IMO.  I already explained why: it requires
me to know up front whether the symbol name I'm about to type is
precise or not.

>     But that's largely immaterial: this bug report is not about the
>     back-end, it's about the UI.  The UI should be independent of the
>     back-end, otherwise the users will be confused when they switch
>     between languages.
> 
> The UI can only work with that a backend returns to it. A non-ideal implementation can result in non-ideal behavior in the end.

We should not have "non-ideal" implementation that return radically
different results.  Each query by default should yield approximately
the same result.  If the differences are small, or differ only in
their order, that's OK.  But having one list of results be 2 orders of
magnitude larger than another is something to avoid at all costs.

>     (If it turns out that some back-ends need to be
>     augmented so that they allow the front-end to present similar UI for
>     the same query, then those back-ends should be enhanced.)
> 
> Sure. I'd be happy to leave that to someone else, but there doesn't seem to be someone actively maintaining it.

That kind of thing happens every day in Emacs development, IME.
There's no way around it, if you care about some component, and
another one gets in the way, you need to fix that other component.

> > To say
> > nothing of the fact that this doesn't scale to any language except
> > ELisp.
> 
> Yes, the Elisp backend doesn't scale to other languages.

I wasn't talking about the elis backend, I was talking about the
design principles.

>     (One of my worst annoyances is to type a
>     C-h command and be presented with "[No match]", because some package
>     is not loaded or some function is not available in the Emacs
>     configuration I happen to be using.)
> 
> It's the cost of doing business, as far as I'm concerned.

No, it's a bug to be fixed.

> You can't use tags for non-core Elisp code anyway, such as anything in your init directory (installed packages, etc), and any Elisp files installed separately by the operating system's distribution.

Of course I can use TAGS: I just need to load those additional tags
tables.

> Actually, if you're not working on a Git checkout, I don't think you can use the tags even for the Elisp code that's part of Emacs.

Why not?  I do it all the time, and tried again just now: it works.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Tue, 28 Apr 2015 22:41:27 -0400
> 
> >> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
> >> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
> >> > and if you type the same one, you get nowhere.  The other matches are
> >> > only available via completion, see below.
> >> Maybe we should write an xref-old-ui-mode.
> > I didn't ask for the old UI.  (If I want it, I can still have it,
> > since the etags.el commands are still available.)  I'm okay with the
> > new UI.  This bug report is about the new UI, and its goal is to make
> > the new UI better.
> 
> Then I don't understand what you mean by the text I quoted.  E.g.:
> 
>    if you try "C-u M-.", you get prompted for the symbol again,
>    and if you type the same one, you get nowhere

If this part of my response gets in the way, just disregard it.
(FWIW, it was an attempt to demonstrate that there's no functionality
in the new UI that is equivalent to "C-u M-." in the old UI.  But it's
not an important point, so feel free to forget about it.)

> > old UI was consistent and complete: it always displayed the first/next
> > match and provided a command to go to the next/previous one.  By
> > contrast, the new UI is inconsistent: with some back-ends it shows the
> > list of matches and allows to navigate it, with others it shows only
> > the first match and does not give any way to get the next match.
> 
> In the "M-. find-tag" example, the UI does let you see all matches.
> It's just that there's only one.

We already agreed that there were at least 2.

More importantly, the issue that I was raising in that example is that
the results change too radically when the back-end is changed.  IMO,
it makes no sense to display more than 140 candidates with one
back-end, and only one with another, in the default configuration.

> If you want, we could make this case popup an *xref* buffer with a
> single entry

Once again, I never said that a single match should be popped in a
buffer.  I said that I don't expect to see a single match in this
case, because that's a lie!

> > But that's largely immaterial: this bug report is not about the
> > back-end, it's about the UI.
> 
> I wouldn't be so sure.

I am.  We keep discussing the back-ends because you and Dmitry
consistently shift the discussion in that direction, so I feel
compelled to respond.  But my perspective in filing the bug report was
the perspective of a user looking at the results through the UI, and
trying to use that UI to manipulate the results.

If some changes are needed in the back-ends to make the UI be better,
then so be it.  But it is not a requirement, and I don't feel I know
enough at this stage to even talk intelligently which changes are
needed and in what parts of this puzzle.

> >> Arguably, the find-tag advice in org-ctags.el could be offered as well,
> >> if org-ctags.el happens to be loaded
> > ??? Why should the xref matches depend on what is and isn't loaded?
> 
> Because that's how xref/elisp works.

If so, then it's much less useful than I thought.  I don't see how
such a back-end can even be a candidate for becoming the default.

> > That would make xref exactly equivalent to "M-x apropos", which means
> > this mode of operation would make xref entirely redundant.
> 
> To you, maybe.  Personally, I find it's *much* quicker to use M-. than
> to use C-h f RET <go-to-the-link> RET

So adding the default guessing depending on what's at point is all
that you need?

> and it has the added benefit that M-, brings me right back if I need
> to.

Apropos pops up a new window, and 'q' quits it, so that problem has a
solution there as well.

> > To say nothing of the fact that this doesn't scale to any language
> > except ELisp.
> 
> Not at all.  Many (most?) packages which offer a functionality along the
> lines of "M-. to jump to the definition" use an implementation technique
> which is fundamentally similar/equivalent (obviously, they don't query
> a running Emacs, but they query a running REPL).

Think about compiled languages.

> > We could offer this mode as an optional feature, but it certainly
> > shouldn't be the default.
> 
> Many users here disagree.

If those many use mainly ELisp or REPL-type language interpreters, I
can understand why.  But Emacs is not limited to those.

> > (One of my worst annoyances is to type a C-h command and be presented
> > with "[No match]", because some package is not loaded or some function
> > is not available in the Emacs configuration I happen to be using.)
> 
> Yes, that's a long standing problem, indeed.  I have a local hack which
> tries to add to loaddefs.el a concise description of where definitions
> might be found (basically, for each file, list the few prefixes used in
> that file), so that "C-h f" can load those packages when needed.
> It's "proof of concept" and it has occasionally rendered my Emacs
> session unusable (e.g. because it ended up deciding to try and load,
> say, w32-fns.el to see if it might contain the definition).  I think
> I've now solved most of those problems in the w32-* files, but the
> underlying risk is still there.

Until that problem is solved in core, I don't see how a back-end that
requires to load a package can be the default.

> > That's one way of looking at it.  Another is to say that etags gave
> > you more information and thus allowed to find more loose matches,
> > which is helpful when your memory is failing you.
> 
> M-. jumps to the definition of the "thing under point".  There's no
> memory involved.

I was talking about "C-u M-."

> If you're not sure what you're looking for, then you're expected to use
> the completion facilities in C-u M-.

Completion shows only part of the matches, we've been through that
already.  It doesn't by default show substring matches.

> > Bonus points for making the filtering optional, since some people
> > might like that, and in some situations even you might need it.
> 
> I disagree.

Then you won't get bonus points from me.

> The filtering is needed by the design of the xref API.
> If you want to see the "unfiltered" data, then use M-x xref-find-apropos.

I didn't say "unfiltered".  I asked for user control of the amount of
filtering or of looseness of the matches.  In etags.el terms, think
about giving the user control on what goes into find-tag-tag-order.

> > Saying that the back-end returns a confusingly large amount of bogus
> > matches, and then switching to a UI that assumes much smarter
> > back-ends (which don't really exist) makes very little sense to me.
> 
> The smarter backends already exist

Where are they?

> in so far as there is already code out there which provides the
> needed functionality.  It just doesn't use the xref API yet, because
> it was written before xref.el.

Ah, so they don't really exist.

Once again, I question the decision to switch to this UI, when we
clearly didn't yet get our act together and are not really prepared to
use it.  The _potential_ to use it exists, but it is currently not
usable.  We should have waited until it was, IMO.

> >   . the UI should depend much less on the back-end, ideally not at all
> 
> You keep repeating this, but that is absolutely meaningless to me (kind
> of like "it doesn't work" in bug reports):
> - This consistency was trivially obtained with the old etags.el code
>   since it had one 1 backend.
> - What the fuck should the new UI do if one backend returns 1 match and
>   another returns a hundred (as in your "find-tag" example)?

It depends on which one is closer to the correct result, obviously.
If 100 is closer, fix or rewrite (or throw away) the back-end that
returns only 1; if 1 is closer, filter the 100 results to return less
than 10.

But having 2 orders of magnitude difference between the results
_by_default_ is absolutely insane!

> >   . there should be a way to go to the next/previous match if the
> >     *xref* buffer is not displayed (or not created in the first
> >     place); if this happens because there's only one match, we should
> >     say so explicitly
> 
> The only case where the *xref* is not displayed is when there's only
> 1 match returned by the backend.  So "go to the next/previous match" is
> again meaningless.

It's not!!  There are partial matches shown by completion -- they are
the next candidates.  If we are not sure the user will want them, we
can ask for confirmation, like we do, for example, in Info's search
that fails to find a match and needs to cross the node boundary.

> >     and should display an indication of the fact that there's only one
> >     match
> 
> The current code "displays" this indication by jumping to the sole match
> instead of jumping to the *xref* buffer.  I think it's clear enough.

We do show similar indications in other cases.  Again, Info's 'i'
comes to mind.

More generally, a UI that does nothing and expects the user to
understand what it means is IMO a bad UI.  At the very least, it
leaves the user wondering whether she hit a bug.

> >   . if the criteria for filtering of the matches are very different
> >     between the possible back-ends, there should be some agreed-upon
> >     uniform default that returns roughly the same number of matches
> >     with all back-ends, and the rest should be controlled by user
> >     options
> 
> I don't see what that could concretely look like.

User options to control how loose the matches are, and the default
level of looseness that yields similar results.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 00:33:13 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: monnier@IRO.UMontreal.CA, 19468@debbugs.gnu.org
> 
>     You could have the few "best matches" listed separately in the
>     beginning of the list, followed by the grouping shown today.
> 
> What's "best matches"? I suggested using `tag-exact-match-p' in the etags backend. What's your take on this?

I'd suggest first an exact match, followed by any matches that are
exact but for letter-case.

Note that I specifically didn't mention tag-exact-match-p, because of
the implicit tag name issue, see below.

>         - Should xref always try to preserve the order in which xrefs are returned?
> 
>     No!  This would make the UI even more dependent on the back-ends.
> 
> Without using the order, in which matches are returned by the backend, we can't even know what the "best matches" are.

Of course, we can: see above.

Moreover, ideally the API to the back-end should allow the UI to
control the matches applied by the back-end, so that the UI gets only
the matches it wants in the first place.

> What about the rest of the predicates in `find-tag-tag-order' set by `etags-recognize-tags-table'? Should we disregard all of them?

Not sure what is included in "the rest".  For example, I don't think
it makes sense to disregard tag-implicit-name-match-p, since many tags
don't have explicit names.

In general, I think it would be good to have a user option that
controls which predicates are used by the etags back-end.  I think we
should group the predicates into meaningful groups (e.g., it makes no
sense to use tag-exact-match-p without tag-implicit-name-match-p).
The default list of the predicates should IMO include these:

  tag-exact-match-p tag-implicit-name-match-p tag-symbol-match-p

File-name matches and word matches are only useful in specialized
searches, and partial matches are just a kind of "fire escape".

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 00:59:46 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: monnier@iro.umontreal.ca, 19468@debbugs.gnu.org
> 
> But anyway, I've just opened lisp/TAGS, and I haven't been able to make `(tag-exact-match-p "...")' return t with any input and position I've tried.

Probably because most tags have implicit names, so you need
tag-implicit-name-match-p.  For exact matches, try edit-abbrevs-map or
allout-item-widget.  See etc/ETAGS.EBNF for more details about
explicit and implicit tag names.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 01:04:44 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>     This feature, including the new UI and API, was introduced 4 months
>     ago, and we still have only 2 back-ends supporting it: ELisp and Ada.
> 
> And etags. Three backends isn't exactly the same as "no backends".

etags is what we had before.  Presumably, xref was introduced because
it was supposed to enable additional functionality, not just wrap and
change the UI of the existing functionality.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 01:07:49 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: monnier@iro.umontreal.ca, 19468@debbugs.gnu.org
> 
>     What alternatives were there, before xref?
> 
> 1. M-x find-function (some even used M-. as its binding).

This is limited to ELisp.

> 2. C-h comment-forward RET, then click on the file name.

This is limited to ELisp and built-in Emacs functions.

I was talking about alternatives that can be used as part of the
development environment suitable for many programming languages.  If
the user needs to switch the "find definition" facility each time she
changes her project or programming language, it's not a good
situation, and therefore such a facility cannot be considered as an
alternative to etags (and ID-utils, if we are talking about finding
references).

>     It's easy to generate TAGS for Lisp files, if one needs them.  We
>     describe all that in the user manual.
> 
> Could you point me to the exact section?

"(info (emacs) Create Tags Table)"

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 01:15:42 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>         There seems some confusion here: you do now invoke xref-find-function. It's not 
>         a function anyway, it's a variable, which different backends can set to 
>         implement the backend interface.
> 
>     Then the confusion is the doc string's fault: it should make this
>     aspect clear.  And if there are no back-ends currently that support
>     these options, I'd suggest to remove that from the doc string -- we
>     shouldn't advertise in the docs stuff that no one implemented.
> 
> The doc string is about the interface: as long as a given backend implements the `references' action, `M-x xref-find-references' will display the returned list.

An interface that has no known implementation doesn't really exist.
I'm asking why do we advertise such a non-existing interface.

> Please feel free to suggest better wording for the docstring.

I already did.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 01:28:53 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: monnier@IRO.UMontreal.CA, 19468@debbugs.gnu.org
> 
>         M-x xref-find-references, for starters.
> 
>     It doesn't seem to work for me: it says "no references found" whatever
>     symbol I try, even those I know are referenced.  What am I missing?
> 
> Do you know how to implement the "all uses" search using only a tags file?

I didn't try that in xref-etags-mode, I tried that with the default
back-end.

> Likewise for Elisp.

Using ID-Utils is one possible way, I guess.  Using Grep is another
(much less efficient one).

> So, sorry to say, both currently implemented backends don't support it.

Too bad.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Wed, 29 Apr 2015 02:24:59 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19468@debbugs.gnu.org
> 
>         That's not so easy to wrap in a global minor mode. I'm not even sure a minor 
>         mode would be a good approach here at all.
> 
>     Sorry, you lost me.  What aspects prevent us from making a global
>     minor mode that uses xref-etags-mode in all buffers?
> 
> Um, yeah, it would work, if you want xref-etags-mode in *all* buffers at all, including those that might use yet another different backend.

Isn't that what a global mode is about?

> Where in the documentation would you mention the key bindings?

In the manual and in NEWS.

>         Should those be `n' and `p' instead, by default? I've found myself using these 
>         bindings very rarely anyway, and `n' is still very close.
> 
>     Possibly.
> 
> I'll wait for a stronger "yes" on this.

Isn't that a rhetorical question, given that 'n' and 'p' are already
bound to those movement commands?

>     There's also "C-x `".  Or maybe you should
>     bring the equivalent of tags-loop-continue back ;-)
> 
> We've already taken up its binding

A negative argument to it could call xref-pop-marker-stack, no?

>         Would you be comfortable with forgetting the current list of errors after 
>         navigating somewhere with xref?
> 
>     No, I don't think so.
> 
> But that what happens if you run a compilation (with errors), and then call Grep. Doesn't it?

No, because Grep and Compilation use different buffers for their
matches.

Re: Etags.el (was: UI inconveniences with M-.)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Tue, 28 Apr 2015 23:12:53 -0400
> 
> >> I really think the etags backend should only return data when the TAGS
> >> file is in one of the parents of the current directory.
> > That goes back to the "working on several projects" discussion of
> > yore.  Specifically, you might be working on two related packages
> > (e.g., one calls the other) that live in two different directories.
> 
> And as mentioned back then, this would be solved by providing a way for
> the user to tell etags.el about the link between those two projects.
> For the case where this link only exists during the running session, we
> could simply use the existing "visit-tags-table" functionality where you
> can add a TAGS file to the one(s) already loaded, so the new
> functionality would behave similarly to the old one.

And as mentioned back then, until something like this is coded, this
discussion is just theoretical.

> >> And it should also be able to keep several independent TAGS tables
> >> opened for different projects in different directory trees.
> > It already does that.
> >
> No it doesn't.  It lets you use the union of a set of TAGS tables, which
> means that the TAGS tables of independent projects pollute each other.

It's an approximation, yes.  Feel free to implement something better;
until then, what we have is quite good IME (I use it every day).

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:44 PM, Eli Zaretskii wrote:

> It depends on which one is closer to the correct result, obviously.
> If 100 is closer, fix or rewrite (or throw away) the back-end that
> returns only 1; if 1 is closer, filter the 100 results to return less
> than 10.
>
> But having 2 orders of magnitude difference between the results
> _by_default_ is absolutely insane!

How would a UI deal with that?

bug#19468: 25.0.50; UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Wed, 29 Apr 2015 18:25:33 +0300 wrote:

 > On 04/29/2015 06:00 PM, Vitalie Spinu wrote:
 >> Let's say I have two projects A and B. By "linked" tag tables I mean
 >> that both tag tables are simultaneously accessible from descendant files
 >> of path/to/A and path/to/B.

 > The etags format already has a similar feature (etags --help | grep include),
 > but it's only one-way, AFAIK.

And if yo have 3,4,... tag tables? And some of them you need to link
only occasionally for a short period of time? I hope you don't insist on
building multiple new tag tables each time one needs a new
"configuration" of tags.

etags.el already supports lists of tag tables. It shouldn't be that
difficult to re-factor it into the proposed "root-local" binding.

  Vitalie

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Vitalie Spinu <spinuvit@gmail.com>
> Date: Wed, 29 Apr 2015 17:00:24 +0200
> Cc: Dmitry Gutov <dgutov@yandex.ru>, 19468@debbugs.gnu.org
> 
> I would like to be able to "link" tag tables.
> 
> Let's say I have two projects A and B. By "linked" tag tables I mean
> that both tag tables are simultaneously accessible from descendant files
> of path/to/A and path/to/B.
> 
> The UI is essentially one command - `etag-link-tag-table`. If I call
> `etag-link-tag-table` from project A and choose a tag table in B then A
> and B tag tables become linked, and vice versa. Simple as that.

The "link" command already exists: just invoke visit-tags-table and
answer YES to the question it asks.  What is missing is the
corresponding "unlink" command, which you didn't mention, but which I
think is required, because at some point you may wish to have Emacs
"forget" about A or B (or both).  (Of course, you can emulate "unlink"
by reloading all the tags tables you need, and answering NO to the
above-mentioned question the first time.  But that's sub-optimal and
requires you to keep notes about which tags tables you have loaded at
all times.)

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 18:57:59 +0300
> 
> On 04/29/2015 06:44 PM, Eli Zaretskii wrote:
> 
> > It depends on which one is closer to the correct result, obviously.
> > If 100 is closer, fix or rewrite (or throw away) the back-end that
> > returns only 1; if 1 is closer, filter the 100 results to return less
> > than 10.
> >
> > But having 2 orders of magnitude difference between the results
> > _by_default_ is absolutely insane!
> 
> How would a UI deal with that?

I don't know yet.  For now, I'm having hard time getting you-all to
agree that this needs to be fixed, and fixed soon.  Are we there yet?

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 07:17 PM, Eli Zaretskii wrote:

>> How would a UI deal with that?
>
> I don't know yet.  For now, I'm having hard time getting you-all to
> agree that this needs to be fixed, and fixed soon.  Are we there yet?

Not really. You seem to insist on fixing this in UI. This isn't far, 
conceptually, from teaching pig how to fly, from where I'm standing.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 19:25:35 +0300
> 
> On 04/29/2015 07:17 PM, Eli Zaretskii wrote:
> 
> >> How would a UI deal with that?
> >
> > I don't know yet.  For now, I'm having hard time getting you-all to
> > agree that this needs to be fixed, and fixed soon.  Are we there yet?
> 
> Not really. You seem to insist on fixing this in UI.

I never said that.  What I did say is that the problems exposed by the
UI need to be fixed, no matter where.

> This isn't far, conceptually, from teaching pig how to fly, from
> where I'm standing.

Thank you for your kind words.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Wed, Apr 29 2015, Stefan Monnier wrote:

> Not sure what that should look like, tho.  Maybe some config var could
> specify which packages's next-error-support is "enabled" by default,
> then some way for the user to override that default on a case by case
> basis: e.g. a command to run inside the *xref* buffer would
> "force-enable" the next-error support for this one time, and maybe
> a new `next-error-pop-context' command would go back to the previous
> "next-error-last-buffer/function"?

Maybe we could mark a buffer somehow that it's a reasonable candidate for
next-error-last-buffer and have a command ala switch-buffer to select
the current next-error-buffer from the list of candidates.

Helmut

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 07:53 PM, Eli Zaretskii wrote:

> I never said that.  What I did say is that the problems exposed by the
> UI need to be fixed, no matter where.

I full-heartedly agree that we should fix problems.

Let me quote you here:

"But that's largely immaterial: this bug report is not about the
back-end, it's about the UI."

"We keep discussing the back-ends because you and Dmitry
consistently shift the discussion in that direction, so I feel
compelled to respond.  But my perspective in filing the bug report was
the perspective of a user looking at the results through the UI"

So you seem to use the term "UI" to refer to the whole user experience 
(as opposed to the UI part of xref, which does not include the 
backends). This might be easier for you, but it leads to confusion, 
because the rest of us are trying to discuss it at a more detailed level.

> Thank you for your kind words.

No offence intended.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 09:10 AM, Helmut Eller wrote:

> Dmitry took that out because he thinks that it interferes too much with
> compilation-mode.  Maybe he's right but let me note that M-x rgrep also
> uses next-error/prev-error and similarly interferes with
> compilation-mode.

Sorry, I guess. I brought up the problem, and didn't see it sufficiently 
addressed or even discussed, so I dropped that part of the code.

Since some discussion is happening how (and users have expressed 
interest in it either way), I'll reinstate the code.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 20:06:07 +0300
> 
> On 04/29/2015 07:53 PM, Eli Zaretskii wrote:
> 
> > I never said that.  What I did say is that the problems exposed by the
> > UI need to be fixed, no matter where.
> 
> I full-heartedly agree that we should fix problems.
> 
> Let me quote you here:
> 
> "But that's largely immaterial: this bug report is not about the
> back-end, it's about the UI."
> 
> "We keep discussing the back-ends because you and Dmitry
> consistently shift the discussion in that direction, so I feel
> compelled to respond.  But my perspective in filing the bug report was
> the perspective of a user looking at the results through the UI"

Where does this say that the problems need to be fixed in the UI?

> So you seem to use the term "UI" to refer to the whole user experience 
> (as opposed to the UI part of xref, which does not include the 
> backends). This might be easier for you, but it leads to confusion, 
> because the rest of us are trying to discuss it at a more detailed level.

I'm sorry that I confused you, and glad that we have that sorted out
now.

> > Thank you for your kind words.
> 
> No offence intended.

You could fool me.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:41 PM, Eli Zaretskii wrote:

> That's a UI inconvenience, IMO.  I already explained why: it requires
> me to know up front whether the symbol name I'm about to type is
> precise or not.

And I have already explained that it has advantages as well.

Interested users can rebind M-. to xref-find-apropos anyway.

>  We should not have "non-ideal" implementation that return radically
> different results.  Each query by default should yield approximately
> the same result.

I agree this should be improved.

> That kind of thing happens every day in Emacs development, IME.
> There's no way around it, if you care about some component, and
> another one gets in the way, you need to fix that other component.

Sure. Or sometimes, you just try to make it work well enough.

> I wasn't talking about the elis backend, I was talking about the
> design principles.

The implementation details of the Elisp backend have nothing to do with 
the other languages.

> No, it's a bug to be fixed.

People are welcome to work on that, but I don't see it work sufficiently 
well and user-friendly in all cases.

>> You can't use tags for non-core Elisp code anyway, such as anything in your init directory (installed packages, etc), and any Elisp files installed separately by the operating system's distribution.
>
> Of course I can use TAGS: I just need to load those additional tags
> tables.

First you have to collect the list of locations, generate TAGS at each 
of them, and then load them. While (okay) it is possible, it's not 
something we can ask the majority of users to perform.

>> Actually, if you're not working on a Git checkout, I don't think you can use the tags even for the Elisp code that's part of Emacs.
>
> Why not?  I do it all the time, and tried again just now: it works.

Okay, you can, if you find out where your distribution installed the 
Elisp files, go there, and generate the TAGS table.

Again, not something I'd ask the average user to do.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 08:15 PM, Eli Zaretskii wrote:

> Where does this say that the problems need to be fixed in the UI?

In "this bug report is ... about the UI".

> I'm sorry that I confused you, and glad that we have that sorted out
> now.

Ok, thanks.

> You could fool me.

Was that a wrong idiom? I only meant to state that "fixing the UI" is 
more or less impossible.

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/29/2015 07:58 PM, Helmut Eller wrote:

> Maybe we could mark a buffer somehow that it's a reasonable candidate for
> next-error-last-buffer and have a command ala switch-buffer to select
> the current next-error-buffer from the list of candidates.

Yes, that sounds like the easiest approach. Then using 
next-error-function in xref at least won't make things worse.

The "mark a buffer somehow that it's a reasonable candidate" is an 
important step, because even plain buffers can have local 
next-error-function values, set by Flycheck or Flymake, for instance.

How will we do that?

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:48 PM, Eli Zaretskii wrote:

> An interface that has no known implementation doesn't really exist.
> I'm asking why do we advertise such a non-existing interface.

There is a known consumer, so implementing it would do something useful. 
Thus, it exists.

> I already did.

Remove? I'm open to other suggestions.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:48 PM, Eli Zaretskii wrote:

> Using ID-Utils is one possible way, I guess.

That's an interesting suggestion. Does it work on all platforms we 
support? Is there some existing Emacs integration?

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@IRO.UMontreal.CA, 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 20:22:14 +0300
> 
> >> Actually, if you're not working on a Git checkout, I don't think you can use the tags even for the Elisp code that's part of Emacs.
> >
> > Why not?  I do it all the time, and tried again just now: it works.
> 
> Okay, you can, if you find out where your distribution installed the 
> Elisp files, go there, and generate the TAGS table.
> 
> Again, not something I'd ask the average user to do.

I don't understand why.  It's a simple matter that is clearly
described in the Emacs user manual, for this very reason.

Quite a few development tools use databases that need to be generated
when you start working on a project, and then regenerated from time to
time.  This is quite routine in any sizable development project, and I
think many developers (a.k.a. "Emacs users") are familiar with this
and won't be intimidated by the need to generate TAGS.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 20:26:18 +0300
> 
> > You could fool me.
> 
> Was that a wrong idiom?

It was an offensive metaphor.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 20:43:30 +0300
> 
> On 04/29/2015 06:48 PM, Eli Zaretskii wrote:
> 
> > Using ID-Utils is one possible way, I guess.
> 
> That's an interesting suggestion. Does it work on all platforms we 
> support?

I'm not aware of a platform where it doesn't.

> Is there some existing Emacs integration?

Yes.  It's called idutils.el, comes with the package, and presents a
Grep-like UI (except that searches are much faster than Grep, and
smarter, of course).

I also see some support in CEDET (judging by file names).

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 08:59 PM, Eli Zaretskii wrote:

> It was an offensive metaphor.

Well, please excuse any offence. It was intended to be humorous.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Wed, Apr 29 2015, Dmitry Gutov wrote:

> The "mark a buffer somehow that it's a reasonable candidate" is an
> important step, because even plain buffers can have local
> next-error-function values, set by Flycheck or Flymake, for instance.
>
> How will we do that?

I would just use a buffer local variable, like next-error-buffer-p, or
perhaps somebody can come up with a more catchy name.

It might be nifty (but more complicated) to organize those buffers as a
stack or ring so that a context can be pushed/popped; if so then it
might be useful to use an integer instead of a boolean as value for this
buffer local variable.

Helmut

Re: UI inconveniences with M-.

[Stefan Monnier]

>> In the "M-. find-tag" example, the UI does let you see all matches.
>> It's just that there's only one.
> We already agreed that there were at least 2.

No, we agreed that in some ideal world there could be two.  But the
backend as it is written now, only returns one for that case.

> More importantly, the issue that I was raising in that example is that
> the results change too radically when the back-end is changed.  IMO,
> it makes no sense to display more than 140 candidates with one
> back-end, and only one with another, in the default configuration.

I agree the 140 "matches" are absurd and need to be fixed.
The single match of the other backend is what I want, OTOH.

> Once again, I never said that a single match should be popped in a
> buffer.  I said that I don't expect to see a single match in this
> case, because that's a lie!

No, it's not a lie and I would even argue it's neither a bug nor
a misfeature.  You expect etags-style information from the xref/elisp
backend, but that's simply not the info it tries to give you.

> compelled to respond.  But my perspective in filing the bug report was
> the perspective of a user looking at the results through the UI, and
> trying to use that UI to manipulate the results.

See, you use the term "UI" in a completely different way.  When I say
"UI" in this discussion, I'm talking about the code which takes the info
from the backend and makes it available to the user.

So the fact that one backend returns 1 item and the other returns 140 of
them is completely outside of the control of the UI.
Maybe the resulting end-user experience sucks, but that's not the fault
of the UI but the fault of the backends.

The reason why I use the term UI in this way is because the API is
designed so that you can have several UIs using the same API (just like
you can have several backends).

>> >> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>> >> if org-ctags.el happens to be loaded
>> > ??? Why should the xref matches depend on what is and isn't loaded?
>> Because that's how xref/elisp works.
> If so, then it's much less useful than I thought.

And to me, it makes it more useful.

> I don't see how such a back-end can even be a candidate for becoming
> the default.

And I don't see how we've live with the etags.el UI for so many years.
I've tried several times to make real use of it, but always found it
completely unpalatable.  What with having to build those damn TAGS
files, remember to refresh them, remember where they are, constantly
tell Emacs again where they are, deal with its inability to find the
right spot and having to repeat the "C-u M-." finger gymnastics
umpteen times.

xref/elisp may not let you jump to those functions/vars that aren't yet
loaded, but beside this little detail, it *just works* without any
extra setup, nothing.

> Apropos pops up a new window, and 'q' quits it, so that problem has a
> solution there as well.

But the apropos is just an intermediate, what I want to see is the
corresponding source, and by the time I'm there, I can't so easily use
`q' to go back.

>> Not at all.  Many (most?) packages which offer a functionality along the
>> lines of "M-. to jump to the definition" use an implementation technique
>> which is fundamentally similar/equivalent (obviously, they don't query
>> a running Emacs, but they query a running REPL).
> Think about compiled languages.

AFAIK, JDEE used the same approach for Java.

[ There's no such thing as "a compiled language".  I guess you mean
  "think of the case where your language's implementation uses good old
  static compilation scheme".  ]

I don't claim that all backends will have to work this way.  Just that
it's a common implementation technique.

> If those many use mainly ELisp or REPL-type language interpreters, I
> can understand why.  But Emacs is not limited to those.

And neither is the xref API.

> Until that problem is solved in core, I don't see how a back-end that
> requires to load a package can be the default.

Agreed.

>> > That's one way of looking at it.  Another is to say that etags gave
>> > you more information and thus allowed to find more loose matches,
>> > which is helpful when your memory is failing you.
>> M-. jumps to the definition of the "thing under point".  There's no
>> memory involved.
> I was talking about "C-u M-."

C-u M-. also lets you do loose matching, via completion, if your memory
is failing you.

>> If you're not sure what you're looking for, then you're expected to use
>> the completion facilities in C-u M-.
> Completion shows only part of the matches, we've been through that
> already.  It doesn't by default show substring matches.

For one, you can make substring the default by tweaking the completion
styles configuration, if you so prefer.  The default completion is
partial-completion, which is actually "more powerful" than substring
matching: e.g. "*find-tag" would not only match "find-tag-tag" and
"my-own-find-tag" but also "my-finder-tag".

But there's nothing magical about substring matching anyway.  etags.el
used it because it was easy to implement, not because it is the best
thing there is.  Of course, it's good, but partial-completion is also
good, and many other options are also good.

> I didn't say "unfiltered".  I asked for user control of the amount of
> filtering or of looseness of the matches.  In etags.el terms, think
> about giving the user control on what goes into find-tag-tag-order.

That's OK, then.  It seems that etags is simply unable to give
sufficiently reliable data to filter it effectively, so we're stuck with
heuristics and user-knobs to choose where they're rather get bitten.

> Once again, I question the decision to switch to this UI, when we

I know, but that's just a waste of time.  The backends won't come before
this system is standard and common place (and they'll take their time,
because in the mean time they can keep using their ad-hoc UI which also
works on older Emacsen).

> But having 2 orders of magnitude difference between the results
> _by_default_ is absolutely insane!

Agreed, because the etags backend sucks right now.

> It's not!!  There are partial matches shown by completion -- they are
> the next candidates.

Again, you choose to look at it this way because you're used to the way
etags.el worked.  But to me, "find-tag" and "find-tag-tag" are not two
different matches to my request.  They're just two completely unrelated
things: either I'm looking for one or I'm looking for the other.

The design of xref is based on the hope that the backend can return
a very small list of candidates, ideally of length 1.  That's not always
the case, but for most potential backends (other than those based on
heuristic techniques like regexp-matching) it's pretty close.

>> >   . if the criteria for filtering of the matches are very different
>> >     between the possible back-ends, there should be some agreed-upon
>> >     uniform default that returns roughly the same number of matches
>> >     with all back-ends, and the rest should be controlled by user
>> >     options
>> I don't see what that could concretely look like.
> User options to control how loose the matches are, and the default
> level of looseness that yields similar results.

Feel free to implement another UI on top of this API which provides this
functionality.  I sure will stay away from it, because I have no use for
something which wastes my time showing me unrelated functions whose name
just happen to contain the name of my function as a substring.

On the contrary, I want a tool that's precise, and gets me directly to
the only corresponding definition (in the 99% of the cases where
there's indeed only one) with a single key-press and no questions asked.


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> Let's say I have two projects A and B. By "linked" tag tables I mean
>> that both tag tables are simultaneously accessible from descendant files
>> of path/to/A and path/to/B.

> The etags format already has a similar feature (etags --help | grep
> include), but it's only one-way, AFAIK.

etags.el also has a feature for this in the `tags-table-list' variable.


        Stefan

Re: Etags.el

[Stefan Monnier]

> And as mentioned back then, until something like this is coded, this
> discussion is just theoretical.

Agreed.  I don't use etags.el so I'm not highly motivated, and when I've
had to look at its code, I've found it has a lot of historical baggage,
which makes it more painful than necessary to do most things (e.g. the
redundancy between tags-table-list and tags-file-name).


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:46 PM, Eli Zaretskii wrote:

> Probably because most tags have implicit names, so you need
> tag-implicit-name-match-p.  For exact matches, try edit-abbrevs-map or
> allout-item-widget.  See etc/ETAGS.EBNF for more details about
> explicit and implicit tag names.

I see, thank you.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:46 PM, Eli Zaretskii wrote:

> I'd suggest first an exact match, followed by any matches that are
> exact but for letter-case.

I think we've settled on "only exact matches" (to the best of backend's 
ability) for xref-find-definitions.

However, in xref-find-apropos we shouldn't like the exact matches that 
much, and likely care about all matches. In that command's output we 
could use the order in which etags returns its results.

>> Without using the order, in which matches are returned by the backend, we can't even know what the "best matches" are.
>
> Of course, we can: see above.

The backend can. The UI can't, thus far (unless only the best matches 
are returned to it).

> Moreover, ideally the API to the back-end should allow the UI to
> control the matches applied by the back-end, so that the UI gets only
> the matches it wants in the first place.

Would you like to describe it in more detail? The current main options 
are: "give me matches for this name" and "give me matches for this 
regexp". There's nothing that would correspond to find-tag-tag-order.

> Not sure what is included in "the rest".  For example, I don't think
> it makes sense to disregard tag-implicit-name-match-p, since many tags
> don't have explicit names.

Okay.

> In general, I think it would be good to have a user option that
> controls which predicates are used by the etags back-end.  I think we
> should group the predicates into meaningful groups (e.g., it makes no
> sense to use tag-exact-match-p without tag-implicit-name-match-p).
> The default list of the predicates should IMO include these:
>
>    tag-exact-match-p tag-implicit-name-match-p tag-symbol-match-p

See the newly added defvar `etags-xref-find-definitions-tag-order'.

The last element seems a bit too lax to me, but it's up to you.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:49 PM, Eli Zaretskii wrote:

> Isn't that what a global mode is about?

If it's what you want, yes. (Nitpick: a "globalized" minor mode is about 
that; a global minor mode can do whatever).

> In the manual and in NEWS.

It would be helpful if you could outline the needed changes in the 
manual: what sections to edit, where, and whether some sections should 
be added or removed.

> Isn't that a rhetorical question, given that 'n' and 'p' are already
> bound to those movement commands?

No, just a stupid one; I forgot they're already bound. But doesn't that 
take care of your concern as well (about `.' and `,' bindings)?

> A negative argument to it could call xref-pop-marker-stack, no?

Won't work for me: adding an extra step to call `xref-pop-marker-stack' 
would be counter-productive.

> No, because Grep and Compilation use different buffers for their
> matches.

Right, but you lose the current errors queue, so to speak, if you call 
Grep in the middle of following Compilation errors.

Re: conflicting uses of next-error-function

[Vitalie Spinu]

 >>> Stefan Monnier on Wed, 29 Apr 2015 09:23:13 -0400 wrote:

 >> Dmitry took that out because he thinks that it interferes too much with
 >> compilation-mode.  Maybe he's right but let me note that M-x rgrep also
 >> uses next-error/prev-error and similarly interferes with
 >> compilation-mode.

 > Indeed, I've occasionally been annoyed by conflicting uses of
 > next-error-last-buffer/next-error-function.  And in both directions:

Shouldn't then the first invocation of next-error push into
xref--marker-ring (for M-.) in case it took you to a wrong place?

 > For that, the next-error framework needs to make it possible for the
 > user to control which package gets to really use next-error and when.
 
How about using the free M-0 prefix in next-error to prompt for the
compilation buffer on which to operate?


   Vitalie

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/30/2015 02:05 AM, Vitalie Spinu wrote:

> Shouldn't then the first invocation of next-error push into
> xref--marker-ring (for M-.) in case it took you to a wrong place?

That would be nice. Something like (unless (eq last-command 'next-error) 
(xref-push-marker-stack)).

> How about using the free M-0 prefix in next-error to prompt for the
> compilation buffer on which to operate?

Unfortunately, it's already taken: both `next-error' and 
`previous-error' take numeric arguments, and `M-0 C-`' means "move by 
zero errors".

Which looks like a waste to me: it pretty rare to move by many errors at 
a time (and know that number precisely).

`C-x 4 C-`'?

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/29/2015 09:15 PM, Helmut Eller wrote:

> It might be nifty (but more complicated) to organize those buffers as a
> stack or ring so that a context can be pushed/popped; if so then it
> might be useful to use an integer instead of a boolean as value for this
> buffer local variable.

Yup, a ring might be a replacement for both next-error-last-buffer and 
next-error-buffer-p. But when will buffers be popped from it?

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/29/2015 04:23 PM, Stefan Monnier wrote:

> and maybe
> a new `next-error-pop-context' command would go back to the previous
> "next-error-last-buffer/function"?

What kind of binding would you see for it?

Re: conflicting uses of next-error-function

[Vitalie Spinu]

 >>> Dmitry Gutov on Thu, 30 Apr 2015 02:11:52 +0300 wrote:

 > On 04/30/2015 02:05 AM, Vitalie Spinu wrote:
 >> Shouldn't then the first invocation of next-error push into
 >> xref--marker-ring (for M-.) in case it took you to a wrong place?

 > That would be nice. Something like (unless (eq last-command 'next-error)
 > (xref-push-marker-stack)).

Yes. But don't offend the `previous-error ;)

 >> How about using the free M-0 prefix in next-error to prompt for the
 >> compilation buffer on which to operate?

 > Unfortunately, it's already taken: both `next-error' and `previous-error' take
 > numeric arguments, and `M-0 C-`' means "move by zero errors".

Hm... I thought that's useless, but it actually makes sense.

 > Which looks like a waste to me: it pretty rare to move by many errors at a time
 > (and know that number precisely).

If after me, numeric prefixes for all navigation commands are pretty a
wasteful. I never use those.

> `C-x 4 C-`'?

It's actually longer "C-x 4 C-x `"'

M-g map is pretty empty. Maybe "M-g e" and M-g M-e? Easy to type on
qwerty.

  Vitalie

(BTW, would be nice to have "M-g c" bound to `goto-char` to keep company
to "M-g g")

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:26 PM, Vitalie Spinu wrote:

> Dmitry, would you mind putting this back?

And done.

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Thu, 30 Apr 2015 03:44:20 +0300 wrote:

 > On 04/29/2015 06:26 PM, Vitalie Spinu wrote:
 >> Dmitry, would you mind putting this back?

 > And done.

Thanks!

Re: conflicting uses of next-error-function

[Helmut Eller]

On Thu, Apr 30 2015, Dmitry Gutov wrote:

> Yup, a ring might be a replacement for both next-error-last-buffer and
> next-error-buffer-p. But when will buffers be popped from it?

Maybe never, or more precisely: dead buffers in the ring are ignored and
removed automatically.

I don't know what the issue with flymake is but it would certainly be
nice if this would work without having to update existing clients.

Helmut

Re: conflicting uses of next-error-function

[Stefan Monnier]

>> and maybe a new `next-error-pop-context' command would go back to the
>> previous "next-error-last-buffer/function"?
> What kind of binding would you see for it?

I was thinking of one of those brain implants.


        Stefan

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/30/2015 09:19 AM, Helmut Eller wrote:

> I don't know what the issue with flymake is but it would certainly be
> nice if this would work without having to update existing clients.

Flycheck (but not Flymake, actually) sets next-error-function is each 
buffer its minor mode is enabled in.

So, we can't simply collect all buffers where next-error-function is 
set, as candidates for `change-next-error-last-buffer'.

Is there anything else I'm missing that we could do without updating 
existing clients?

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Wed, 29 Apr 2015 17:54:04 -0400
> 
> >> In the "M-. find-tag" example, the UI does let you see all matches.
> >> It's just that there's only one.
> > We already agreed that there were at least 2.
> 
> No, we agreed that in some ideal world there could be two.  But the
> backend as it is written now, only returns one for that case.

If the back-end conceals potentially useful information, it should be
fixed not to do so.

> > More importantly, the issue that I was raising in that example is that
> > the results change too radically when the back-end is changed.  IMO,
> > it makes no sense to display more than 140 candidates with one
> > back-end, and only one with another, in the default configuration.
> 
> I agree the 140 "matches" are absurd and need to be fixed.
> The single match of the other backend is what I want, OTOH.

Only when we are 100% sure there is really only one match relevant to
the user.  When we aren't 100% sure, it's better to show a few
additional candidates and rely on the user to disregard them if she
doesn't need them.

IOW, a bit of additional, perhaps redundant information is much less
of a problem than missing information.

That you personally find that information always redundant does not
mean it's true for everyone else.  There's more than one use case for
these queries, see below.

> > Once again, I never said that a single match should be popped in a
> > buffer.  I said that I don't expect to see a single match in this
> > case, because that's a lie!
> 
> No, it's not a lie and I would even argue it's neither a bug nor
> a misfeature.  You expect etags-style information from the xref/elisp
> backend, but that's simply not the info it tries to give you.

Once again, if the back-end conceals potentially useful information,
the back-end needs to be fixed.

While etags shows a very long list of potential matches, it produces
them in the order of importance; I don't remember ever having to
examine more than the first few hits (and thus wasn't even aware of of
the sheer length of the full list).  So with the old UI the
ridiculously long list of candidates was never in my face.  The new
UI, by contrast, makes this acutely evident by showing me the full
list, and on top of that it destroys the order produced by the
back-end, which would allow me to look only at the few first hits.

IOW, the etags back-end by default produces a long list because it
relies on the old front-end to avoid annoying the user by the length
of that list.  The new UI, by contrast, is clearly designed on the
assumption that the list of candidates is very short, so it must
either limit the number of match strategies used by etags, or filter
the results according to some useful criteria.

> > compelled to respond.  But my perspective in filing the bug report was
> > the perspective of a user looking at the results through the UI, and
> > trying to use that UI to manipulate the results.
> 
> See, you use the term "UI" in a completely different way.  When I say
> "UI" in this discussion, I'm talking about the code which takes the info
> from the backend and makes it available to the user.

That's your misunderstanding.  I described my user experience from
using this new feature; I never said where the fixes should be made,
primarily because I didn't understand the code well enough (and still
don't) to say anything intelligent about it.

If I wanted to talk about the code, I'd say something like
"this-and-that function does wrong things because of so-and-so".

> So the fact that one backend returns 1 item and the other returns 140 of
> them is completely outside of the control of the UI.

Says you.  Through my naive-user eyes, filtering 140 hits to provide
just a few is perfectly within the capabilities of the UI, at least in
principle.

> Maybe the resulting end-user experience sucks, but that's not the fault
> of the UI but the fault of the backends.

It sounds more and more like it could be the fault of the design, and
specifically of how functionality is divided between the back-end and
the UI.  Let me elaborate.

From the rest of the discussion, and the related threads spawned by
it, I seem to get the feeling that the design was explicitly meant to
leave the UI completely oblivious to the subject matter, and instead
infinitely trust the back-end to produce "the right" results.  If so,
I think this is a design mistake, and this discussion (and my
frustrating user experience) is the result.  IME, an interactive
user-level feature that supports multiple back-end engines cannot
avoid some degree of subject-matter knowledge and some kind of
manipulation of the results that come from the back-end, or else it
won't be able to provide consistent user experience across the
supported back-ends.  Having such a "dumb" UI in this case will cause
complications, some of which we already see:

  . the need to "fix" the back-ends, and continue "fixing" them in the
    future
  . the need to merge back-ends and invent some suitable
    infrastructure for that
  . etc.

To me, this means that separating the back-ends from the UI while
leaving the UI "dumb" is basically unworkable, because such a
separation does not really separate anything: there will still be a
very high degree of coherency and cohesion between the two parts.  Or
maybe there will only be one usable back-end, and the rest will
bit-rot.

IOW, the separation of functionality is in the wrong place.  To be
useful, some of the "smarts" need to be on the UI side, where user
control can be best implemented, and where user intent is known much
better.  Then we could stop caring about back-ends providing too much
info, because the filtering will be in the UI and, most importantly,
by uniform criteria.  And even if the filtering is in some of the
back-ends, it will be according to UI's guidance, whereby the UI will
convert the user-level control options into the language
understandable by the back-end.  That is the only practical way of
providing a consistent user experience with multiple back-ends.

> >> >> Arguably, the find-tag advice in org-ctags.el could be offered as well,
> >> >> if org-ctags.el happens to be loaded
> >> > ??? Why should the xref matches depend on what is and isn't loaded?
> >> Because that's how xref/elisp works.
> > If so, then it's much less useful than I thought.
> 
> And to me, it makes it more useful.
> 
> > I don't see how such a back-end can even be a candidate for becoming
> > the default.
> 
> And I don't see how we've live with the etags.el UI for so many years.

How frequently did you need to study a large body of code written by
someone else, like a large project where you need to fix a bug?

I do that a lot, both on my daytime job, and in Free Software related
moonlighting.  If you seldom, or never, have to quickly find your way
in such vast unchartered territories, and mainly work with code that
either you yourself wrote or with which you are intimately familiar,
then I can understand why you don't like more than one match
candidate.

Likewise, people who mainly work with small projects, where the code
line count is in hundreds or a small number of thousands, might never
need the less-than-exact matches provided by etags: Grep is good
enough and fast enough for that.  But you cannot do without etags in a
larger project.  (You also cannot do without being able to find all
references to an identifier, which is why I'm using ID Utils for the
past 15 years, and was disappointed to learn that xref-find-references
has no back-ends that implemented this kind of search.)

I very much hope Emacs will continue to be able to support the kind of
activities I described above, which AFAIK are very important part of a
software developer's job throughout the industry.

> I've tried several times to make real use of it, but always found it
> completely unpalatable.  What with having to build those damn TAGS
> files, remember to refresh them, remember where they are, constantly
> tell Emacs again where they are, deal with its inability to find the
> right spot and having to repeat the "C-u M-." finger gymnastics
> umpteen times.

Those are exaggerations.  Building TAGS is almost instant, even in
Emacs, you need only refresh them very seldom, and Emacs offers the
place from which to load them so you don't need to remember.

But this, again, is immaterial for this discussion.  I hope you will
agree that, whatever issues we have with etags, replacing it with
something that lacks important functionality is not a good idea.

> xref/elisp may not let you jump to those functions/vars that aren't yet
> loaded, but beside this little detail, it *just works* without any
> extra setup, nothing.

That "little detail" all but invalidates most of my use cases.  We
seem to have very different development experiences, probably due to
what each one of us does most of the time, but I hope you can see that
there are valid use cases outside of your personal experience, and
Emacs should support them if it wants to be a good IDE.

And I'm not even sure your ideas of how to solve that "little detail"
are workable, because of the potentially adverse consequences of
loading code you don't actually need (or want) to execute.  What if
the code is buggy, or dangerous, or simply does things you don't want
to be done in your Emacs session?

And how do you scale this up to packages that are outside of
load-path, for whatever reasons?  Etc. etc.

> >> Not at all.  Many (most?) packages which offer a functionality along the
> >> lines of "M-. to jump to the definition" use an implementation technique
> >> which is fundamentally similar/equivalent (obviously, they don't query
> >> a running Emacs, but they query a running REPL).
> > Think about compiled languages.
> 
> AFAIK, JDEE used the same approach for Java.

I don't know what that means, and don't know enough about JDEE to talk
about it.  In any case, Java is not a "classic" compiled language, as
a Jave development environment is generally capable of running the
code in an interpreter.

> I don't claim that all backends will have to work this way.  Just that
> it's a common implementation technique.

Common for some paradigms of development activity, for some
development tasks, and probably also for some programming languages.
It's not universally true by any measure.

> > Until that problem is solved in core, I don't see how a back-end that
> > requires to load a package can be the default.
> 
> Agreed.

But since it was already made the default, it means we must fix this
ASAP.

> >> > That's one way of looking at it.  Another is to say that etags gave
> >> > you more information and thus allowed to find more loose matches,
> >> > which is helpful when your memory is failing you.
> >> M-. jumps to the definition of the "thing under point".  There's no
> >> memory involved.
> > I was talking about "C-u M-."
> 
> C-u M-. also lets you do loose matching, via completion, if your memory
> is failing you.

I don't think completion is the right tool for these searches, because
the name alone doesn't tell enough.  So if we want to base that on
name-completion, we are asking the user to type "something TAB" every
time she needs another candidate.  I don't think this is a very
convenient UI for this job.

> > I didn't say "unfiltered".  I asked for user control of the amount of
> > filtering or of looseness of the matches.  In etags.el terms, think
> > about giving the user control on what goes into find-tag-tag-order.
> 
> That's OK, then.  It seems that etags is simply unable to give
> sufficiently reliable data to filter it effectively, so we're stuck with
> heuristics and user-knobs to choose where they're rather get bitten.

No.  etags provided you a long list, but ordered by importance.  As I
said, I don't remember ever having to examine more than the few first
candidates.  etags relied on this order to save the user from having
to deal with a huge list of potential candidates, at least in most
cases.  etags _can_ effectively filter the results it produces, if you
ask it to use less members in the find-tag-tag-order list.  Or you
could just take the first N candidates of the list it returns, and
ignore the rest.

> > Once again, I question the decision to switch to this UI, when we
> 
> I know, but that's just a waste of time.  The backends won't come before
> this system is standard and common place (and they'll take their time,
> because in the mean time they can keep using their ad-hoc UI which also
> works on older Emacsen).

See above: you assume that the division of functionality between the
UI and the back-ends is at the right place.  I'm not so sure.  If I'm
right, then when more back-ends come, we will see more problems, not
less.

> > It's not!!  There are partial matches shown by completion -- they are
> > the next candidates.
> 
> Again, you choose to look at it this way because you're used to the way
> etags.el worked.

Not to how etags.el worked, to how _I_ worked, what _I_ needed to do
with etags.el's help.  We simply have 2 very different kinds of jobs
to do most of our time.  One more argument for more user control on
the UI level, if you ask me.

> But to me, "find-tag" and "find-tag-tag" are not two different
> matches to my request.  They're just two completely unrelated
> things: either I'm looking for one or I'm looking for the other.

Assuming you know what you are looking for, yes.  I described a
situation that is frequent for me where you generally don't, at least
not well enough to be satisfied by a single exact match.

> The design of xref is based on the hope that the backend can return
> a very small list of candidates, ideally of length 1.

And therein lies its weakness.  I actually don't understand how this
kind of assumption could be allowed to exist, when the _default_
back-end, and one of only 3 existing ones, blows that assumption out
of the water.

> On the contrary, I want a tool that's precise, and gets me directly to
> the only corresponding definition (in the 99% of the cases where
> there's indeed only one) with a single key-press and no questions asked.

Yes, I want that too.  I also want a tool which will write code for
me.  I just don't believe either of these is possible, because in most
of my use cases (which I believe are shared by many in the industry)
no program can ever be lucky enough to guess what definition is that.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@IRO.UMontreal.CA, 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Thu, 30 Apr 2015 01:56:40 +0300
> 
> On 04/29/2015 06:46 PM, Eli Zaretskii wrote:
> 
> > I'd suggest first an exact match, followed by any matches that are
> > exact but for letter-case.
> 
> I think we've settled on "only exact matches" (to the best of backend's 
> ability) for xref-find-definitions.

I believe the default in Emacs is to use caseless searches in these
situations, and leave it to the user to either customize
case-fold-search or type the exact letter-case if she wants such exact
matches.  With languages that customarily use both letter-cases, I see
no reason to deviate from that practice.

> However, in xref-find-apropos we shouldn't like the exact matches that 
> much, and likely care about all matches.

Yes.

> In that command's output we could use the order in which etags
> returns its results.

I agree.

> >> Without using the order, in which matches are returned by the backend, we can't even know what the "best matches" are.
> >
> > Of course, we can: see above.
> 
> The backend can. The UI can't, thus far (unless only the best matches 
> are returned to it).

If it can't, it's probably because no one coded that.  But the rules
are not so complex, so it's not inconceivable that such code could
exist in the UI.

> > Moreover, ideally the API to the back-end should allow the UI to
> > control the matches applied by the back-end, so that the UI gets only
> > the matches it wants in the first place.
> 
> Would you like to describe it in more detail? The current main options 
> are: "give me matches for this name" and "give me matches for this 
> regexp". There's nothing that would correspond to find-tag-tag-order.

Why not learn from find-tag-tag-order, and allow the same categories
of matches as it uses, sans those that make no sense outside of the
TAGS data?

> > In general, I think it would be good to have a user option that
> > controls which predicates are used by the etags back-end.  I think we
> > should group the predicates into meaningful groups (e.g., it makes no
> > sense to use tag-exact-match-p without tag-implicit-name-match-p).
> > The default list of the predicates should IMO include these:
> >
> >    tag-exact-match-p tag-implicit-name-match-p tag-symbol-match-p
> 
> See the newly added defvar `etags-xref-find-definitions-tag-order'.
> 
> The last element seems a bit too lax to me, but it's up to you.

Thanks, I will take a look soon.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Thu, 30 Apr 2015 02:03:50 +0300
> 
> > Isn't that a rhetorical question, given that 'n' and 'p' are already
> > bound to those movement commands?
> 
> No, just a stupid one; I forgot they're already bound. But doesn't that 
> take care of your concern as well (about `.' and `,' bindings)?

I hope so, yes.

> > No, because Grep and Compilation use different buffers for their
> > matches.
> 
> Right, but you lose the current errors queue, so to speak, if you call 
> Grep in the middle of following Compilation errors.

Maybe I didn't try too hard, but it didn't happen to me when I did.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Thu, Apr 30 2015, Dmitry Gutov wrote:

> Is there anything else I'm missing that we could do without updating
> existing clients?

next-error calls next-error-find-buffer.  Maybe we could record every
buffer that was returned by next-error-find-buffer and include them in
the candidate list.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/30/2015 04:48 PM, Eli Zaretskii wrote:

> I believe the default in Emacs is to use caseless searches in these
> situations, and leave it to the user to either customize
> case-fold-search or type the exact letter-case if she wants such exact
> matches.  With languages that customarily use both letter-cases, I see
> no reason to deviate from that practice.

Yeah, that still works like that. "Followed by any matches ... but for 
letter-case" simply means to me to use case-insensitive matching in the 
first place, since the backend doesn't return matches in groups.

However, the kind of matches could be included in the "location group" name.

Or we could even allow groups to nest somehow (similarly to 
semantic-symref output, which nests tags within functions, within 
files). However, this idea alone won't preserve the ordering of groups.

> If it can't, it's probably because no one coded that.  But the rules
> are not so complex, so it's not inconceivable that such code could
> exist in the UI.

Still, that doesn't sound like a good idea. Backends know their 
languages better.

> Why not learn from find-tag-tag-order, and allow the same categories
> of matches as it uses, sans those that make no sense outside of the
> TAGS data?

 From where I'm standing, that's a customization preference, not a 
design suggestion. The latter would include a proposal of changes to the 
xref-find-function interface.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 1 May 2015 05:27:18 +0300
> 
> On 04/30/2015 04:48 PM, Eli Zaretskii wrote:
> 
> > I believe the default in Emacs is to use caseless searches in these
> > situations, and leave it to the user to either customize
> > case-fold-search or type the exact letter-case if she wants such exact
> > matches.  With languages that customarily use both letter-cases, I see
> > no reason to deviate from that practice.
> 
> Yeah, that still works like that. "Followed by any matches ... but for 
> letter-case" simply means to me to use case-insensitive matching in the 
> first place, since the backend doesn't return matches in groups.

I'd still suggest to place the exact matches first.  The UI could do
that.

> > If it can't, it's probably because no one coded that.  But the rules
> > are not so complex, so it's not inconceivable that such code could
> > exist in the UI.
> 
> Still, that doesn't sound like a good idea. Backends know their 
> languages better.

I don't see anything language-specific here.  Etags.el has no
language-specific knowledge, either (it is delegated to etags.c).

Like I said elsewhere, I think making the UI too dumb might give us
sub-optimal design, wrt how functionality is divided between the UI
and the back-ends, and in particular produce more of unhappy user
experiences caused by the UI relying on the back-ends too much, and
the back-ends differing between them too much.

Case sensitivity, partial matches, and other similar stuff employed by
etags, and conceptually by any other similar facility, has almost
nothing to do with languages.  And if (and when) we want to support
match rules that do depend on languages, the API should allow
specifying such rules to the back-end, and the back-end should DTRT
when it receives a spec it cannot support.  For example, if the UI
requests only exact letter-case match, and the back-end is for a
case-insensitive language, that back-end should produce caseless
matches nonetheless.

> > Why not learn from find-tag-tag-order, and allow the same categories
> > of matches as it uses, sans those that make no sense outside of the
> > TAGS data?
> 
>  From where I'm standing, that's a customization preference, not a 
> design suggestion. The latter would include a proposal of changes to the 
> xref-find-function interface.

I stopped short of providing a complete proposal.  My design
suggestion is to generalize and abstract the match types used by
etags.  Customization preferences might then control which subset of
the set of match types are used for each query.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/01/2015 09:45 AM, Eli Zaretskii wrote:

> I don't see anything language-specific here.  Etags.el has no
> language-specific knowledge, either (it is delegated to etags.c).

elis-mode.el does. In this case, it's using find-func.el, which uses the 
information we have about Elisp environment at runtime.

Here's another example:

The point is on a foo.bar() call in a Java-like language. Or maybe C++.

bar is present in classes A, B and C.

If the parser knows the type of foo to be A, the backend employing it 
can bring us to the one definition. If, however, the backend returns the 
xrefs for A#foo, B#foo and C#foo, there's no way the UI could 
distinguish between them.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 1 May 2015 14:27:40 +0300
> 
> On 05/01/2015 09:45 AM, Eli Zaretskii wrote:
> 
> > I don't see anything language-specific here.  Etags.el has no
> > language-specific knowledge, either (it is delegated to etags.c).
> 
> elis-mode.el does. In this case, it's using find-func.el, which uses the 
> information we have about Elisp environment at runtime.

Sorry, I don't understand what "Elisp environment at runtime" means in
practice, or how it's used to affect what results are returned for a
query.

> The point is on a foo.bar() call in a Java-like language. Or maybe C++.
> 
> bar is present in classes A, B and C.
> 
> If the parser knows the type of foo to be A, the backend employing it 
> can bring us to the one definition. If, however, the backend returns the 
> xrefs for A#foo, B#foo and C#foo, there's no way the UI could 
> distinguish between them.

That's the case where the UI should instruct the back-end what it
needs, because the back-end doesn't know which of these alternatives
is the right one.  If the user wants all bar functions, or maybe those
whose parent class matches some regexp, not just the one from the
class instance at point, then producing only one match would be wrong,
and the UI won't be able to correct that.

Re: UI inconveniences with M-.

[Stefan Monnier]

> If the back-end conceals potentially useful information, it should be
> fixed not to do so.

"Conceal" makes it sound like it's ill-intentioned, done on purpose
or something.

For (defadvice find-tag ...), the problem is in advice.el.
Patches welcome, but since this library is on the way out AFAIC, I'd
recommend you don't bother spending time on it.

> That you personally find that information always redundant does not
> mean it's true for everyone else.  There's more than one use case for
> these queries, see below.

We can't know what the user wants and handle all conceivable scenarios.
Maybe the user really wants to jump to this chunk of code somewhere that
does (fset foo bar) and hence ends up overriding the "official"
definition of the function.  But unless you can solve the halting
problem and read the user's mind at the same time, we'll have to settle
for something imperfect.

etags.el also failed miserably in some scenarios which xref/elisp
handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.

> That's your misunderstanding.  I described my user experience from
> using this new feature; I never said where the fixes should be made,

Well, you kept insisting that it's not a question of backend, so maybe
you didn't say where the fixes should go, but you did say where the blame
should go.

> Says you.  Through my naive-user eyes, filtering 140 hits to provide
> just a few is perfectly within the capabilities of the UI, at least in
> principle.

The 140 hits are just a list of locations.  The UI has no fucking clue
whether these are function definitions or what, nor does it know in
which language the file is written.

> IOW, the separation of functionality is in the wrong place.  To be
> useful, some of the "smarts" need to be on the UI side, where user
> control can be best implemented, and where user intent is known much
> better.

The UI has to be agnostic.  So the smarts can't be in the UI.  The API
can be extended to provide the extra smarts that the UI might need, of
course.  E.g. we could add to the API a function that sorts/groups the
entries, so the etags backend can sort them based on "likelyhood" rather
than group them by file.

> I very much hope Emacs will continue to be able to support the kind of
> activities I described above, which AFAIK are very important part of a
> software developer's job throughout the industry.

You fail to understand why complaint about etags.el.  I'm not
complaining about `etags', but about the etags.el front-end, which is in
need of improvement to handle the case where the user is navigating
several completely different projects and doesn't want one to pollute
the other one.

>> I've tried several times to make real use of it, but always found it
>> completely unpalatable.  What with having to build those damn TAGS
>> files, remember to refresh them, remember where they are, constantly
>> tell Emacs again where they are, deal with its inability to find the
>> right spot and having to repeat the "C-u M-." finger gymnastics
>> umpteen times.
> Those are exaggerations.

Partly, yes.  I'm just venting my frustration with the tool, and
pointing out that if xref/elisp (and xref/etags) has some downsides,
etags.el had its own set of downsides.  And some of those shouldn't be
that hard to fix (tho they would probably be a lot easier if we didn't
have to worry about annoying some old-time users because they'd have to
slightly change their habits).

> Building TAGS is almost instant, even in Emacs,

The problem is not computer-time but human-time.

> you need only refresh them very seldom, and Emacs offers the
> place from which to load them so you don't need to remember.

If Emacs knows where the file is, the user shouldn't need to be queried.

> But this, again, is immaterial for this discussion.  I hope you will
> agree that, whatever issues we have with etags, replacing it with
> something that lacks important functionality is not a good idea.

As I said, going back to etags.el is not an option.

> That "little detail" all but invalidates most of my use cases.

Then don't use that backend.  E.g. use xref-etags-mode.

>> C-u M-. also lets you do loose matching, via completion, if your memory
>> is failing you.
> I don't think completion is the right tool for these searches, because
> the name alone doesn't tell enough.

Don't pretend you don't know about xref-apropos.

> See above: you assume that the division of functionality between the
> UI and the back-ends is at the right place.

No I don't.  I don't assume the API is fixed either.  All I assume is
that the UI can't know about the programming language or about the
quality of any given answer, or any such thing.

>> But to me, "find-tag" and "find-tag-tag" are not two different
>> matches to my request.  They're just two completely unrelated
>> things: either I'm looking for one or I'm looking for the other.
> Assuming you know what you are looking for, yes.  I described a
> situation that is frequent for me where you generally don't, at least
> not well enough to be satisfied by a single exact match.

And that's not what M-. for.  For that we have xref-apropos.

> And therein lies its weakness.  I actually don't understand how this
> kind of assumption could be allowed to exist, when the _default_

Because this assumption is known to be obtainable, with help from the
toolchain (the compiler will generally know with 100% certainty the few
possible definitions matching a particular use).


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/01/2015 03:57 PM, Eli Zaretskii wrote:

> Sorry, I don't understand what "Elisp environment at runtime" means in
> practice, or how it's used to affect what results are returned for a
> query.

find-func knows about defined functions and defined variables. 
elisp-mode knows that a function usually goes after a paren, and a 
variable - after a space (to simplify things).

Thus, elisp-xref-find could narrow the search space based on whether 
there is a paren before the symbol at point (we don't do that, partially 
because the situation is more complicated; but we should, in the 
future). A language-agnostic UI won't ever be able to do so.

> That's the case where the UI should instruct the back-end what it
> needs, because the back-end doesn't know which of these alternatives
> is the right one.  If the user wants all bar functions, or maybe those
> whose parent class matches some regexp, not just the one from the
> class instance at point, then producing only one match would be wrong,
> and the UI won't be able to correct that.

If the user calls xref-find-definitions, we consider that to see the 
definition of the function called at point (or definitions, if virtual 
dispatch is unavoidable, such as in case of a Java interface, and there 
are several options), but not more.

For more lax matching options, the user will call xref-find-apropos.

bug#19468: 25.0.50; UI inconveniences with M-.

[Stefan Monnier]

>> bar is present in classes A, B and C.
>> If the parser knows the type of foo to be A, the backend employing it 
>> can bring us to the one definition. If, however, the backend returns the 
>> xrefs for A#foo, B#foo and C#foo, there's no way the UI could 
>> distinguish between them.
> That's the case where the UI should instruct the back-end what it
> needs, because the back-end doesn't know which of these alternatives
> is the right one.

You got it backwards: here, only the backend (which knows about the
language's rules) can know, by asking the compiler.  Remember: when you
hit M-., the backend is asked to find the definition of the entity under
point (so it has the complete context of occurrence of "bar" and can
figure out that it's looking for the "bar" of class "A"), not the
definition of a context-deprived "bar" identifier.


        Stefan

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Fri, 01 May 2015 10:21:04 -0400
> 
> > If the back-end conceals potentially useful information, it should be
> > fixed not to do so.
> 
> "Conceal" makes it sound like it's ill-intentioned, done on purpose
> or something.

Not according to my references.

> > That you personally find that information always redundant does not
> > mean it's true for everyone else.  There's more than one use case for
> > these queries, see below.
> 
> We can't know what the user wants and handle all conceivable scenarios.

Isn't that what I said?

> Maybe the user really wants to jump to this chunk of code somewhere that
> does (fset foo bar) and hence ends up overriding the "official"
> definition of the function.  But unless you can solve the halting
> problem and read the user's mind at the same time, we'll have to settle
> for something imperfect.

Emacs has a school-book solution for these problems: let the user
specify what she wants, with prefix arguments and user options.

My point was that a back-end that isn't prepared for such degree of
control, and decides for the user what she wants is not just
"imperfect", it's semi-broken.

> etags.el also failed miserably in some scenarios which xref/elisp
> handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.

I don't see diff-mode-abrev-table in TAGS, so I don't understand what
you want to demonstrate with that.

> > That's your misunderstanding.  I described my user experience from
> > using this new feature; I never said where the fixes should be made,
> 
> Well, you kept insisting that it's not a question of backend, so maybe
> you didn't say where the fixes should go, but you did say where the blame
> should go.

No, I said that blaming the back-end for what I observed, and saying
that nothing can be done "because that's what the back-end gives" is
not the right way of approaching these kinds of problems.

> > Says you.  Through my naive-user eyes, filtering 140 hits to provide
> > just a few is perfectly within the capabilities of the UI, at least in
> > principle.
> 
> The 140 hits are just a list of locations.  The UI has no fucking clue
> whether these are function definitions or what, nor does it know in
> which language the file is written.

The 140 hits are symbol names, not just locations.  Discerning between
exact matches, caseless matches, and partial matches does not need any
language-specific knowledge.  I don't understand what problems you see
here.

But if you are certain that the fixes belong to the back-end, it's
fine with me.  Just don't tell me that the back-end is to blame, and
therefore there's nothing that can be done.

> > IOW, the separation of functionality is in the wrong place.  To be
> > useful, some of the "smarts" need to be on the UI side, where user
> > control can be best implemented, and where user intent is known much
> > better.
> 
> The UI has to be agnostic.  So the smarts can't be in the UI.

The question is which smarts.  I'm saying that the UI cannot blindly
trust the back-end to always DTRT, it needs to control it according to
its needs and user preferences.  These smarts _must_ be in the UI.
Having just 2 or 3 query types -- either get me only one match or give
me all of them -- is too rough, there must be more shades of gray, and
more ways for the user to extend and refine the query.

The old API provided an approximation to this by returning a long list
in order of importance, so where you stopped asking for more was a
rather crude, but usually effective way of getting exactly what you
wanted.  This is now gone, so we must find other, hopefully better
ways of giving the user the same fine-grained control on what she gets
in response to a query.

> > I very much hope Emacs will continue to be able to support the kind of
> > activities I described above, which AFAIK are very important part of a
> > software developer's job throughout the industry.
> 
> You fail to understand why complaint about etags.el.  I'm not
> complaining about `etags', but about the etags.el front-end, which is in
> need of improvement to handle the case where the user is navigating
> several completely different projects and doesn't want one to pollute
> the other one.

This issue is tangential to the subject f this discussion.  My
complaints are valid even if you work with a single project, and in
fact were observed while working with a single project.

> >> I've tried several times to make real use of it, but always found it
> >> completely unpalatable.  What with having to build those damn TAGS
> >> files, remember to refresh them, remember where they are, constantly
> >> tell Emacs again where they are, deal with its inability to find the
> >> right spot and having to repeat the "C-u M-." finger gymnastics
> >> umpteen times.
> > Those are exaggerations.
> 
> Partly, yes.  I'm just venting my frustration with the tool, and
> pointing out that if xref/elisp (and xref/etags) has some downsides,
> etags.el had its own set of downsides.

Irrelevant.  I wasn't saying etags was better, or didn't have
disadvantages.  I was talking about deficiencies of the current
solution, as a whole, for looking up definitions of symbols.  I don't
really care which back-end is used for that, but I do care about the
results, and I do care when if I switch back-ends and suddenly get
very different results.

> (tho they would probably be a lot easier if we didn't have to worry
> about annoying some old-time users because they'd have to slightly
> change their habits).

Yes, I'm an annoying guy.  Feel free to ask me to step down and
disappear from here, if that annoys you too much.  (The profanities
are already a sign of that.)

> > Building TAGS is almost instant, even in Emacs,
> 
> The problem is not computer-time but human-time.

What human time?  The time it takes to type "make TAGS"?

> > you need only refresh them very seldom, and Emacs offers the
> > place from which to load them so you don't need to remember.
> 
> If Emacs knows where the file is, the user shouldn't need to be queried.

Emacs can guess where the user _might_ want it, but it has to allow
for seldom exceptions, otherwise how can the user switch to another
project or add its data to the current one?

> > But this, again, is immaterial for this discussion.  I hope you will
> > agree that, whatever issues we have with etags, replacing it with
> > something that lacks important functionality is not a good idea.
> 
> As I said, going back to etags.el is not an option.

It's definitely an option for this curmudgeon, if the new-and-improved
solution will not become better.

> > That "little detail" all but invalidates most of my use cases.
> 
> Then don't use that backend.  E.g. use xref-etags-mode.

Are we again talking about my personal problems?  If they are only
mine, I can solve them very easily, and don't need this discussion or
the bug reports to do so.

What do you tell users whose use case is similar to mine?  "Use
xref-etags-mode"?  Then why did we switch away of etags.el, if we
still need to use its core machinery?

> >> C-u M-. also lets you do loose matching, via completion, if your memory
> >> is failing you.
> > I don't think completion is the right tool for these searches, because
> > the name alone doesn't tell enough.
> 
> Don't pretend you don't know about xref-apropos.

I don't want 2 different commands.  I know about etags-apropos as
well, but never used it, because I never needed to.  Forcing me to use
2 different commands where I could use one is an annoyance.

Anyway, I'm out of this discussion.  Good luck selling your ideas and
use cases to others.  I'm not sold.  And I don't enjoy having to read
uncalled-for profanities in response to legitimate questions and
issues.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 1 May 2015 17:51:14 +0300
> 
> On 05/01/2015 03:57 PM, Eli Zaretskii wrote:
> 
> > Sorry, I don't understand what "Elisp environment at runtime" means in
> > practice, or how it's used to affect what results are returned for a
> > query.
> 
> find-func knows about defined functions and defined variables. 
> elisp-mode knows that a function usually goes after a paren, and a 
> variable - after a space (to simplify things).
> 
> Thus, elisp-xref-find could narrow the search space based on whether 
> there is a paren before the symbol at point (we don't do that, partially 
> because the situation is more complicated; but we should, in the 
> future). A language-agnostic UI won't ever be able to do so.

A language-agnostic UI could well ask the back-end for variables, or
for functions, or for both, or whatever.

Like I said: some functionality must reside in the back-end, but the
UI must control it, instead of blindly trusting its defaults.  When we
blindly trust the defaults of each back-end, we get what triggered
this discussion, because etags' default is to produce a 140-long list
of potential matches, which elisp-mode's xref default is to produce
only one.  In most of my use cases, neither is TRT.

> > That's the case where the UI should instruct the back-end what it
> > needs, because the back-end doesn't know which of these alternatives
> > is the right one.  If the user wants all bar functions, or maybe those
> > whose parent class matches some regexp, not just the one from the
> > class instance at point, then producing only one match would be wrong,
> > and the UI won't be able to correct that.
> 
> If the user calls xref-find-definitions, we consider that to see the 
> definition of the function called at point (or definitions, if virtual 
> dispatch is unavoidable, such as in case of a Java interface, and there 
> are several options), but not more.

We shall never second-guess the user.  I already described an
important class of use cases where this assumption is simply wrong.
It shouldn't probably even be the default.

> For more lax matching options, the user will call xref-find-apropos.

It's an annoyance to have to use more than one command for a single
purpose.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/01/2015 09:38 PM, Eli Zaretskii wrote:

> A language-agnostic UI could well ask the back-end for variables, or
> for functions, or for both, or whatever.

Why would it ask about "functions". How would it know about functions, 
or that we want a function right now?

> because etags' default is to produce a 140-long list
> of potential matches, which elisp-mode's xref default is to produce
> only one.  In most of my use cases, neither is TRT.

That is no longer true. You should build the current master.

> We shall never second-guess the user.  I already described an
> important class of use cases where this assumption is simply wrong.
> It shouldn't probably even be the default.

Either I didn't understand it, or wasn't convinced.

> It's an annoyance to have to use more than one command for a single
> purpose.

By default, I don't want to see the list at all, most of the time, just 
jump to the only match. We won't have that if xref-find-definitions is lax.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 1 May 2015 21:44:12 +0300
> 
> On 05/01/2015 09:38 PM, Eli Zaretskii wrote:
> 
> > A language-agnostic UI could well ask the back-end for variables, or
> > for functions, or for both, or whatever.
> 
> Why would it ask about "functions". How would it know about functions, 
> or that we want a function right now?

If it doesn't, it should ask the user.

> > because etags' default is to produce a 140-long list
> > of potential matches, which elisp-mode's xref default is to produce
> > only one.  In most of my use cases, neither is TRT.
> 
> That is no longer true. You should build the current master.

The example is still valid.  How do you know another back-end won't do
something similar?

> > It's an annoyance to have to use more than one command for a single
> > purpose.
> 
> By default, I don't want to see the list at all, most of the time, just 
> jump to the only match. We won't have that if xref-find-definitions is lax.

Note the "I" vs the "we".  If _you_ want to see only one match most of
the time, you should be able to customize the feature to do just that.
Others could customize it differently, according to their use cases.
It will still be the same command, though.

Other IDEs ask the user explicitly to specify how wide she wants the
search and how detailed the results.  We could do that as well,
although I think it's less Emacsy.

But having just one level is never right in such cases.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/01/2015 10:22 PM, Eli Zaretskii wrote:

> If it doesn't, it should ask the user.

That's a speed bump. Not to say that a generic symbol-browsing facility 
handling different kinds of them won't be useful, but that's not what I 
see `M-.' doing.

> The example is still valid.  How do you know another back-end won't do
> something similar?

How do you know another back-end won't return 140 results for "function 
named car"? In the end, it's up to the backend's author to write an 
implementation conforming to what's being asked of it.

> Note the "I" vs the "we".  If _you_ want to see only one match most of
> the time, you should be able to customize the feature to do just that.
> Others could customize it differently, according to their use cases.
> It will still be the same command, though.

There's nothing to note. While *I* can only speak for myself, Stefan 
expressed a similar preference. IIRC, both Jorgen and Helmut did so as well.

Of course, I'd be happy to make it more customizable, if it can still 
work well enough for me, at least in one configuration. But I'll need 
more concrete design proposals on that.

> Other IDEs ask the user explicitly to specify how wide she wants the
> search and how detailed the results.  We could do that as well,
> although I think it's less Emacsy.

That hasn't been my experience. They do have different commands for 
different searches, but those vary between editors. Yet, there's always 
a "go to definition" command that tries its best not to ask the user 
anything else, and jumps to wherever the symbol at point was "defined". 
It's usually bound to Ctrl-Click.

> But having just one level is never right in such cases.

I wonder if it's even possible to define a set of levels in a 
backend-agnostic way. For instance, for etags you have "exact matches", 
"exact implicit matches", "word matches", "partial file name matches", etc.

In the Elisp backend, I'd see the "same kind of entity", "other kinds of 
entities with that name", "fuzzy matches on the name".

Keep in mind though, that if we introduce the notion of levels in the 
interface, it'll also complicate things for every implementor.

Maybe the solution is to define the ability for a backend to return 
groups, probably nested ones.

Re: UI inconveniences with M-.

[Stefan Monnier]

>> etags.el also failed miserably in some scenarios which xref/elisp
>> handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.
> I don't see diff-mode-abrev-table in TAGS, so I don't understand what
> you want to demonstrate with that.

Exactly that: etags fails to find the definition site of
diff-mode-abrev-table, whereas xref/elisp does find it.

>> As I said, going back to etags.el is not an option.
> It's definitely an option for this curmudgeon, if the new-and-improved
> solution will not become better.

Of course you can.  I'm only talking about the default.

> What do you tell users whose use case is similar to mine?  "Use
> xref-etags-mode"?  Then why did we switch away of etags.el, if we
> still need to use its core machinery?

We do want to support the etags backend for a long time to come.
The old etags.el UI on the other hand, will most likely be phased out
(or rewritten on top of the new API no it can also be used with other
backends).

> I don't want 2 different commands.  I know about etags-apropos as
> well, but never used it, because I never needed to.  Forcing me to use
> 2 different commands where I could use one is an annoyance.

It's a tradeoff.  The upside is that M-. gets you there much quicker
then M-. RET C-u M-. C-u M-., and that xref-apropos gives you a quick
overview of all the matches, compared to cycling through C-u M-. where
you never know how long the cycle will be nor where it will take you next.

> Anyway, I'm out of this discussion.  Good luck selling your ideas and
> use cases to others.  I'm not sold.  And I don't enjoy having to read
> uncalled-for profanities in response to legitimate questions and
> issues.

I've found you didn't have many constructive suggestions and spent too
much time complaining about how the new system doesn't work like the
old one.
That's frustrating.


        Stefan

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/30/2015 04:42 PM, Eli Zaretskii wrote:

> While etags shows a very long list of potential matches, it produces
> them in the order of importance; I don't remember ever having to
> examine more than the first few hits (and thus wasn't even aware of of
> the sheer length of the full list).  So with the old UI the
> ridiculously long list of candidates was never in my face.  The new
> UI, by contrast, makes this acutely evident by showing me the full
> list, and on top of that it destroys the order produced by the
> back-end, which would allow me to look only at the few first hits.

That is unfortunate indeed. Browsing the xref-find-apropos results 
segregated by the match distance might prove useful.

> If I wanted to talk about the code, I'd say something like
> "this-and-that function does wrong things because of so-and-so".

I think both I and Stefan very much wanted you to look at the actual 
code here. It's much easier to make demands on functionality than to 
propose a clean and modular design.

> Through my naive-user eyes, filtering 140 hits to provide
> just a few is perfectly within the capabilities of the UI, at least in
> principle.

That is, of course, possible. But then we'll need to define some 
universal language-arnostic metadata that the UI would use to operate.

> It sounds more and more like it could be the fault of the design, and
> specifically of how functionality is divided between the back-end and
> the UI.  Let me elaborate.
>
> <...>
> To me, this means that separating the back-ends from the UI while
> leaving the UI "dumb" is basically unworkable, because such a
> separation does not really separate anything: there will still be a
> very high degree of coherency and cohesion between the two parts.  Or
> maybe there will only be one usable back-end, and the rest will
> bit-rot.

Sorry, all I see here is a lot of conjecture. The current 
implementations took not a lot of code, and they work well enough.

You seem to argue in favor of extracting some common code into the UI, 
but there's not much of it that I can see. Again, it would help if your 
proposals were more technical and lower-level.

> Likewise, people who mainly work with small projects, where the code
> line count is in hundreds or a small number of thousands, might never
> need the less-than-exact matches provided by etags: Grep is good
> enough and fast enough for that.  But you cannot do without etags in a
> larger project.  (You also cannot do without being able to find all
> references to an identifier, which is why I'm using ID Utils for the
> past 15 years, and was disappointed to learn that xref-find-references
> has no back-ends that implemented this kind of search.)

M-x xref-find-references is implemented now, for both backends, using 
Grep or one of the faster tools (semantic/symref makes the choice), and 
the "symbol bounds" matching. Hope you're happier now.

By the way, CEDET has had a similar feature for a while (try `M-x 
semantic-mode', then `C-c , G' on some function, in a C file). Arguably, 
even with a better interface.

Any reason why you haven't been using it?

> That "little detail" all but invalidates most of my use cases.  We
> seem to have very different development experiences, probably due to
> what each one of us does most of the time, but I hope you can see that
> there are valid use cases outside of your personal experience, and
> Emacs should support them if it wants to be a good IDE.

I'm sure we want it to be. However, I've seen no indication that the 
majority of users prefers find-tag over find-function. Maybe some hard 
data would change that.

> And I'm not even sure your ideas of how to solve that "little detail"
> are workable, because of the potentially adverse consequences of
> loading code you don't actually need (or want) to execute.  What if
> the code is buggy, or dangerous, or simply does things you don't want
> to be done in your Emacs session?

That's a valid concern, but you'd have to read that kind of project from 
top to bottom first. Then you can load it and proceed to use the 
navigation functions.

In any case, we should probably return to this example after it occurs 
in our ecosystem at least once.

> I don't know what that means, and don't know enough about JDEE to talk
> about it.  In any case, Java is not a "classic" compiled language, as
> a Jave development environment is generally capable of running the
> code in an interpreter.

There are several ones for C, example: 
http://www.drdobbs.com/cpp/ch-a-cc-interpreter-for-script-computing/184402054

Not that it has any relevance to the xref backend interface.

> But since it was already made the default, it means we must fix this
> ASAP.

-1.

> I don't think completion is the right tool for these searches, because
> the name alone doesn't tell enough.  So if we want to base that on
> name-completion, we are asking the user to type "something TAB" every
> time she needs another candidate.  I don't think this is a very
> convenient UI for this job.

Indeed, xref-find-apropos is better for that.

> See above: you assume that the division of functionality between the
> UI and the back-ends is at the right place.  I'm not so sure.  If I'm
> right, then when more back-ends come, we will see more problems, not
> less.

That's conjecture again.

> Assuming you know what you are looking for, yes.  I described a
> situation that is frequent for me where you generally don't, at least
> not well enough to be satisfied by a single exact match.

I'd like to repeat my suggestion:

- You can bind `M-.' to `xref-find-apropos' in your config.
- We can improve its output somehow.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 12:04 AM, Stefan Monnier wrote:

> then M-. RET C-u M-. C-u M-., and that xref-apropos gives you a quick

You seem to be subtly suggesting that the command should be renamed.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 09:03 PM, Eli Zaretskii wrote:

>>> Using ID-Utils is one possible way, I guess.

It will be used now, courtesy of semantic/symref, if the ID file is in 
the same directory as TAGS.

Now someone should document that, without forgetting GNU Global and 
CScope, as well as the fact that their index files tend to get out of date.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 03:08:39 +0300
> 
> On 04/29/2015 09:03 PM, Eli Zaretskii wrote:
> 
> >>> Using ID-Utils is one possible way, I guess.
> 
> It will be used now, courtesy of semantic/symref, if the ID file is in 
> the same directory as TAGS.

Thanks.  I think it's a good idea to add an ID target to the top-level
Makefile.in.  Or does Semantic run that automatically as needed?

> Now someone should document that, without forgetting GNU Global and 
> CScope, as well as the fact that their index files tend to get out of date.

Yep.  At least mkid is lightning-fast (10 sec on the entire Emacs
tree), so updating ID is not an annoyance IME.

Re: UI inconveniences with M-.

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

> Maybe the solution is to define the ability for a backend to return groups,
> probably nested ones.

I don't see much need for nesting.  I think the issue is mostly one of
sorting in *xref* (or in the cycling, if we were to use a hypothetical
other UI that provides functionality like the old etags.el).

So, let's focus on this.
My suggestion would be either:
- make xref-find-function return not just a list of locations, but also
  an indication whether they're pre-sorted or not.
- make xref-find-function return not a list of locations, but a list of
  lists of locations (i.e. every location inside a sub-list is
  considered to have the same "level/quality/likelihood", but the lists
  are sorted by their "level/quality/likelihood").
- let the backend provide its own sorting function to override the
  current "group by file" sorting.

Basically, I think the focus should be on making sure that we can
re-implement the old behavior on top of the new API when using the
xref/etags backend.

Another thing that we should consider is how to let the user choose
which backend to use.  Currently we only have xref-etags-mode, but
I think it would make sense to make it possible for the user to
dynamically choose among the potentially many more backends (e.g. we
could add an xref/grep backend, and there will hopefulyl be more using
things like id-utils, gtags, ...) and maybe also to use several backends
at the same time (merging the results).  I don't have any suggestions
for what this could/should look like, OTOH.  But maybe one way to do
that would be to let each backend expose via the API a set of knobs
(one of which would be an "enable/disable" knob), and then provide an
xref command that can manage these knobs conveniently.


        Stefan

Re: UI inconveniences with M-.

[Stefan Monnier]

>> then M-. RET C-u M-. C-u M-., and that xref-apropos gives you a quick
> You seem to be subtly suggesting that the command should be renamed.

No, I'm just lazy,


        Stefan

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 03:08:39 +0300
> 
> On 04/29/2015 09:03 PM, Eli Zaretskii wrote:
> 
> >>> Using ID-Utils is one possible way, I guess.
> 
> It will be used now, courtesy of semantic/symref, if the ID file is in 
> the same directory as TAGS.

I cannot seem to make it to work.  Here's what I did:

  ./src/emacs -Q
  C-x C-f src/xdisp.c RET
  move to a place that references set_cursor_from_row
  M-x xref-find-references RET

I get "Symbol's function definition is void: mapcan".

I then manually load cl, and re-invoke the command.  The result now is
"No known references for: set_cursor_from_row".  But the shell command
"gid set_cursor_from_row" returns this list of matches:

  src/xdisp.c:174:   latter, see also set_cursor_from_row).
  src/xdisp.c:380:   cursor; <= 0 if not known.  Set by set_cursor_from_row, used for
  src/xdisp.c:777:static bool set_cursor_from_row (struct window *, struct glyph_row *,
  src/xdisp.c:1406:                set_cursor_from_row conspires with cursor_row_p to
  src/xdisp.c:4421:                 text, which confuses `set_cursor_from_row'.
  src/xdisp.c:13675:            set_cursor_from_row (w, row, w->current_matrix, 0, 0, 0, 0);
  src/xdisp.c:14191:set_cursor_from_row (struct window *w, struct glyph_row *row,
  src/xdisp.c:15568:                set_cursor_from_row (w, row, w->current_matrix, 0, 0, 0, 0);
  src/xdisp.c:15582:               occlude point.  We need to let set_cursor_from_row
  src/xdisp.c:15596:                  rv |= set_cursor_from_row (w, row, w->current_matrix,
  src/xdisp.c:15652:                if (set_cursor_from_row (w, row, w->current_matrix, 0, 0, 0, 0))
  src/xdisp.c:16202:        set_cursor_from_row (w, row, w->desired_matrix, 0, 0, 0, 0);
  src/xdisp.c:16634:      set_cursor_from_row (w, row, w->current_matrix, 0, 0, 0, 0);
  src/xdisp.c:17081:              set_cursor_from_row (w, row, w->current_matrix, 0, 0,
  src/xdisp.c:17317:               bidi-reordered glyph rows.  Let set_cursor_from_row
  src/xdisp.c:17322:                if (!set_cursor_from_row (w, row, w->current_matrix,
  src/xdisp.c:17892:          set_cursor_from_row (w, row, current_matrix, 0, 0, 0, 0);
  src/xdisp.c:17932:          set_cursor_from_row (w, row, current_matrix, 0, 0, 0, 0);
  src/xdisp.c:18154:          set_cursor_from_row (w, row, w->current_matrix, 0, 0, 0, 0);
  src/xdisp.c:18163:          set_cursor_from_row (w, row, w->current_matrix, delta,
  src/xdisp.c:20794:    set_cursor_from_row (it->w, row, it->w->desired_matrix, 0, 0, 0, 0);

In case it matters, the ID file was produced manually with this
command in the top-level Emacs directory:

  mkid -o ID .

What am I missing?

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Fri, 01 May 2015 17:04:48 -0400
> 
> > Anyway, I'm out of this discussion.  Good luck selling your ideas and
> > use cases to others.  I'm not sold.  And I don't enjoy having to read
> > uncalled-for profanities in response to legitimate questions and
> > issues.
> 
> I've found you didn't have many constructive suggestions and spent too
> much time complaining about how the new system doesn't work like the
> old one.
> That's frustrating.

Please find a more civilized way of expressing your frustration.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 1 May 2015 23:36:31 +0300
> 
> On 05/01/2015 10:22 PM, Eli Zaretskii wrote:
> 
> > If it doesn't, it should ask the user.
> 
> That's a speed bump.

Yes.  But the alternative, of producing output that the user deosn't
want, or no output at all, is worse, don't you think?

> How do you know another back-end won't return 140 results for "function 
> named car"? In the end, it's up to the backend's author to write an 
> implementation conforming to what's being asked of it.

Yes, but the specification to which it should conform should be more
detailed, otherwise some back-end could very well claim conformance
and still produce such large lists.  The restrictions, such as "only
exact matches" or "only functions" etc. should be a necessary part of
the API -- then and only then would it be possible to reject a
back-end that returns 140 hits for "exact matches only" query.

> > Note the "I" vs the "we".  If _you_ want to see only one match most of
> > the time, you should be able to customize the feature to do just that.
> > Others could customize it differently, according to their use cases.
> > It will still be the same command, though.
> 
> There's nothing to note. While *I* can only speak for myself, Stefan 
> expressed a similar preference. IIRC, both Jorgen and Helmut did so as well.

So at best you have 80% of users that would want the same as you.
What about the other 20%?  Shouldn't we provide customizable options
for them, even though they might be a minority?

> Of course, I'd be happy to make it more customizable, if it can still 
> work well enough for me, at least in one configuration. But I'll need 
> more concrete design proposals on that.

My proposal was to come up with types of matches (I mentioned them
already several times), and then let the users customize the
collection of types passed to the back-end by default.  Whether this
translates immediately to some design, I don't know, although it
seemed to me it does.

> I wonder if it's even possible to define a set of levels in a 
> backend-agnostic way.

I think it is.

> For instance, for etags you have "exact matches", 
> "exact implicit matches", "word matches", "partial file name matches", etc.

Except for "implicit matches" that should actually be part of the same
generalized class as "exact matches", the rest are quite general, I
think.

> In the Elisp backend, I'd see the "same kind of entity", "other kinds of 
> entities with that name", "fuzzy matches on the name".

"Fuzzy" is "substring" in etags, "same/other kinds" should be
translated to "functions/variables" etc.

> Keep in mind though, that if we introduce the notion of levels in the 
> interface, it'll also complicate things for every implementor.

The goal is to provide a better user experience, not to make life
easier for the implementors.

But if a certain back-end can do a good job by implementing only one
level, into which all the levels will map, then it's an okay solution
for that back-end's targets.

Re: UI inconveniences with M-.

[Helmut Eller]

On Sat, May 02 2015, Stefan Monnier wrote:

> - let the backend provide its own sorting function to override the
>   current "group by file" sorting.

The current grouping preserves the order: xrefs are grouped by the
string returned by xref-location-group and both the order of the groups
and the order of xrefs within the group is preserved. See
xref--alistify.

> Basically, I think the focus should be on making sure that we can
> re-implement the old behavior on top of the new API when using the
> xref/etags backend.

Theoretically this can already be done by ignoring xref-location-group
or by specializing xref-location-group for xref-etags-location so that it
returns always the same group.

Thinking about it: for etags it would make sense to group xrefs by name
of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
"globally"; xrefs are reordered only within a TAGS file.

Helmut

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 00:11:28 +0300
> 
> > If I wanted to talk about the code, I'd say something like
> > "this-and-that function does wrong things because of so-and-so".
> 
> I think both I and Stefan very much wanted you to look at the actual 
> code here. It's much easier to make demands on functionality than to 
> propose a clean and modular design.

One person can only do so much, given her free time, and can only be
an expert in so many fields.  When you or Stefan report a problem with
the display engine, or in some other area where I know enough, I don't
ask for a design before I start working on it.  In this case, all I
have to offer is user experience, some requirements for what I
consider to be a good solution, and some general guidelines for such a
solution (which I only provided in response to repeated demands to do
so).  If that is not useful, perhaps we should revise our instructions
for bug reporting.

IOW, I was reporting problems in an area where I know very little.  I
don't think it's fair to demand that I provide, let alone code, a
solution, as a prerequisite for acting on my report.  I think the job
of making the feature better in response to bug reports is the job of
those who worked on the feature to begin with, and thus have intimate
knowledge of the design and the implementation.

> > Through my naive-user eyes, filtering 140 hits to provide
> > just a few is perfectly within the capabilities of the UI, at least in
> > principle.
> 
> That is, of course, possible. But then we'll need to define some 
> universal language-arnostic metadata that the UI would use to operate.

I think this kind of universal metadata is very much clear and mostly
already supported.  After all, Emacs has been doing this kind of jobs
for a very long time, which is why we have notions like e.g. "symbols",
which each language defines differently.

> > It sounds more and more like it could be the fault of the design, and
> > specifically of how functionality is divided between the back-end and
> > the UI.  Let me elaborate.
> >
> > <...>
> > To me, this means that separating the back-ends from the UI while
> > leaving the UI "dumb" is basically unworkable, because such a
> > separation does not really separate anything: there will still be a
> > very high degree of coherency and cohesion between the two parts.  Or
> > maybe there will only be one usable back-end, and the rest will
> > bit-rot.
> 
> Sorry, all I see here is a lot of conjecture.

Of course it is.  What else did you expect from a bystander who wasn't
involved in the design and implementation?

It is up to you to do whatever you want with this conjecture.  Some
people pay a lot of money to get my conjectures in this and similar
fields.  You get it for free.

> The current implementations took not a lot of code, and they work
> well enough.

That's not an evidence of the design validity, not IME.  This feature
is with us for too short time to be able to draw conclusions about its
design.  At least it "didn't work well enough" for me, which is a sign
that it isn't perfect.  And you already made quite a few changes based
on my experience.  I think it might be a good time to step back and
review those changes, looking for some more fundamental issues that
might benefit from some redesign.  I don't know what you will find, if
you do that, but I do know that if you continue to insist that the
design is perfect, you will never see its flaws, if they exist.

> By the way, CEDET has had a similar feature for a while (try `M-x 
> semantic-mode', then `C-c , G' on some function, in a C file). Arguably, 
> even with a better interface.
> 
> Any reason why you haven't been using it?

Partly because CEDET is too heavyweight for most of my needs, and
partly because I simply didn't have enough time to learn it.

> > And I'm not even sure your ideas of how to solve that "little detail"
> > are workable, because of the potentially adverse consequences of
> > loading code you don't actually need (or want) to execute.  What if
> > the code is buggy, or dangerous, or simply does things you don't want
> > to be done in your Emacs session?
> 
> That's a valid concern, but you'd have to read that kind of project from 
> top to bottom first. Then you can load it and proceed to use the 
> navigation functions.

That's impossible.  I'm talking about projects whose line counts are
in hundreds of thousands, sometimes millions.  You cannot read such
project from top to bottom, when all you need to do is fix some bug or
find the reason for some particular behavior: you will never make your
deadline.  Using the tools I'm talking about is the only way to find
the _relevant_ places which you do need to read and understand.

IOW, your methodology would put the cart before the horse, in these
use cases.

> > I don't know what that means, and don't know enough about JDEE to talk
> > about it.  In any case, Java is not a "classic" compiled language, as
> > a Jave development environment is generally capable of running the
> > code in an interpreter.
> 
> There are several ones for C, example: 
> http://www.drdobbs.com/cpp/ch-a-cc-interpreter-for-script-computing/184402054
> 
> Not that it has any relevance to the xref backend interface.

Then why bring it up?

> > See above: you assume that the division of functionality between the
> > UI and the back-ends is at the right place.  I'm not so sure.  If I'm
> > right, then when more back-ends come, we will see more problems, not
> > less.
> 
> That's conjecture again.

And I have some gray heir to back it up with.  I have learned from
long experience that good design is frequently backed up by intuition
and "conjecture".  I'm not necessarily saying I'm right in this case,
but what if I am?  Shouldn't you at least consider that, instead of
rejecting it flatly as "conjecture" and sticking to the original
design as if it were a scripture?

Re: UI inconveniences with M-.

[Fumitaka Tokumitsu]

[-- Attachment #1: Type: text/plain, Size: 7848 bytes --]

さほと



—
Sent from Mailbox

On Wed, Apr 29, 2015 at 11:41 AM, Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> [ Moving this to emacs-devel.  ]
>>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
>>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
>>> > and if you type the same one, you get nowhere.  The other matches are
>>> > only available via completion, see below.
>>> Maybe we should write an xref-old-ui-mode.
>> I didn't ask for the old UI.  (If I want it, I can still have it,
>> since the etags.el commands are still available.)  I'm okay with the
>> new UI.  This bug report is about the new UI, and its goal is to make
>> the new UI better.
> Then I don't understand what you mean by the text I quoted.  E.g.:
>    if you try "C-u M-.", you get prompted for the symbol again,
>    and if you type the same one, you get nowhere
> Well, duh!  What did you expect?  In the new UI, "C-u M-." jumps to
> *the* definition (the "C-u" is only used to let you type the name of
> thing you're looking for), so of course if you just jumped to it, asking
> to jump to it again will get you nowhere else.  I guess it would be OK
> to make M-. emit a message like "Hello?  We're already here!" in this
> corner case, if you want to make sure something happens.
>> old UI was consistent and complete: it always displayed the first/next
>> match and provided a command to go to the next/previous one.  By
>> contrast, the new UI is inconsistent: with some back-ends it shows the
>> list of matches and allows to navigate it, with others it shows only
>> the first match and does not give any way to get the next match.
> In the "M-. find-tag" example, the UI does let you see all matches.
> It's just that there's only one.  If you want, we could make this case
> popup an *xref* buffer with a single entry, but I think it makes more
> sense to just jump straight to that one entry instead.
>> But that's largely immaterial: this bug report is not about the
>> back-end, it's about the UI.
> I wouldn't be so sure.
>>> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>>> if org-ctags.el happens to be loaded
>> ??? Why should the xref matches depend on what is and isn't loaded?
> Because that's how xref/elisp works.
>> That would make xref exactly equivalent to "M-x apropos", which means
>> this mode of operation would make xref entirely redundant.
> To you, maybe.  Personally, I find it's *much* quicker to use M-. than
> to use C-h f RET <go-to-the-link> RET, and it has the added benefit that
> M-, brings me right back if I need to.
>> To say nothing of the fact that this doesn't scale to any language
>> except ELisp.
> Not at all.  Many (most?) packages which offer a functionality along the
> lines of "M-. to jump to the definition" use an implementation technique
> which is fundamentally similar/equivalent (obviously, they don't query
> a running Emacs, but they query a running REPL).
>> We could offer this mode as an optional feature, but it certainly
>> shouldn't be the default.
> Many users here disagree.
>> (One of my worst annoyances is to type a C-h command and be presented
>> with "[No match]", because some package is not loaded or some function
>> is not available in the Emacs configuration I happen to be using.)
> Yes, that's a long standing problem, indeed.  I have a local hack which
> tries to add to loaddefs.el a concise description of where definitions
> might be found (basically, for each file, list the few prefixes used in
> that file), so that "C-h f" can load those packages when needed.
> It's "proof of concept" and it has occasionally rendered my Emacs
> session unusable (e.g. because it ended up deciding to try and load,
> say, w32-fns.el to see if it might contain the definition).  I think
> I've now solved most of those problems in the w32-* files, but the
> underlying risk is still there.
>> That's one way of looking at it.  Another is to say that etags gave
>> you more information and thus allowed to find more loose matches,
>> which is helpful when your memory is failing you.
> M-. jumps to the definition of the "thing under point".  There's no
> memory involved.
> If you're not sure what you're looking for, then you're expected to use
> the completion facilities in C-u M-.
>> But in the context of this bug report, that, too, is immaterial: if we
>> think the etags back-end gives too much information, by all means
>> let's filter it before presenting the matches.
> Yes, that's clearly very much needed.
>> Bonus points for making the filtering optional, since some people
>> might like that, and in some situations even you might need it.
> I disagree.  The filtering is needed by the design of the xref API.
> If you want to see the "unfiltered" data, then use M-x xref-find-apropos.
>> Then let's fix the results we display with the etags back-end to show
>> only the relevant ones.
> Yes, please.
>> Saying that the back-end returns a confusingly large amount of bogus
>> matches, and then switching to a UI that assumes much smarter
>> back-ends (which don't really exist) makes very little sense to me.
> The smarter backends already exist, in so far as there is already code out
> there which provides the needed functionality.  It just doesn't use the
> xref API yet, because it was written before xref.el.
>> The old code attempted to support both use cases, by showing the exact
>> match(es) first, followed by less likely ones.  We do similar things
>> in other commands.  The advantages of such a method are that (1) you
>> don't need to second-guess the user, (2) you provide only one command
>> for the user to remember, and (3) if the user only _thinks_ she knows
>> the name, but really doesn't, she can find the information without
>> having to invoke another command.
>>> I think that's an important feature of the new code in this respect.
>> But it is incomplete without a means to get to the other possible
>> matches in this case.
>> To summarize my points in this sub-thread:
>>   . the UI should depend much less on the back-end, ideally not at all
> You keep repeating this, but that is absolutely meaningless to me (kind
> of like "it doesn't work" in bug reports):
> - This consistency was trivially obtained with the old etags.el code
>   since it had one 1 backend.
> - What the fuck should the new UI do if one backend returns 1 match and
>   another returns a hundred (as in your "find-tag" example)?
>>   . there should be a way to go to the next/previous match if the
>>     *xref* buffer is not displayed (or not created in the first
>>     place); if this happens because there's only one match, we should
>>     say so explicitly
> The only case where the *xref* is not displayed is when there's only
> 1 match returned by the backend.  So "go to the next/previous match" is
> again meaningless.
>>   . when there are more than one possible match, the UI should behave
>>     similarly, i.e. display the *xref* buffer; when there's only one
>>     match, it should _never_ display *xref*,
> That's exactly what the code does.
>>     and should display an indication of the fact that there's only one
>>     match
> The current code "displays" this indication by jumping to the sole match
> instead of jumping to the *xref* buffer.  I think it's clear enough.
>>   . if the criteria for filtering of the matches are very different
>>     between the possible back-ends, there should be some agreed-upon
>>     uniform default that returns roughly the same number of matches
>>     with all back-ends, and the rest should be controlled by user
>>     options
> I don't see what that could concretely look like.
>         Stefan

[-- Attachment #2: Type: text/html, Size: 8951 bytes --]

Re: UI inconveniences with M-.

[Fumitaka Tokumitsu]

[-- Attachment #1: Type: text/plain, Size: 5883 bytes --]

にた泣かねけあせけかきねはきかきか



—
Sent from Mailbox

On Fri, May 1, 2015 at 11:21 PM, Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

>> If the back-end conceals potentially useful information, it should be
>> fixed not to do so.
> "Conceal" makes it sound like it's ill-intentioned, done on purpose
> or something.
> For (defadvice find-tag ...), the problem is in advice.el.
> Patches welcome, but since this library is on the way out AFAIC, I'd
> recommend you don't bother spending time on it.
>> That you personally find that information always redundant does not
>> mean it's true for everyone else.  There's more than one use case for
>> these queries, see below.
> We can't know what the user wants and handle all conceivable scenarios.
> Maybe the user really wants to jump to this chunk of code somewhere that
> does (fset foo bar) and hence ends up overriding the "official"
> definition of the function.  But unless you can solve the halting
> problem and read the user's mind at the same time, we'll have to settle
> for something imperfect.
> etags.el also failed miserably in some scenarios which xref/elisp
> handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.
>> That's your misunderstanding.  I described my user experience from
>> using this new feature; I never said where the fixes should be made,
> Well, you kept insisting that it's not a question of backend, so maybe
> you didn't say where the fixes should go, but you did say where the blame
> should go.
>> Says you.  Through my naive-user eyes, filtering 140 hits to provide
>> just a few is perfectly within the capabilities of the UI, at least in
>> principle.
> The 140 hits are just a list of locations.  The UI has no fucking clue
> whether these are function definitions or what, nor does it know in
> which language the file is written.
>> IOW, the separation of functionality is in the wrong place.  To be
>> useful, some of the "smarts" need to be on the UI side, where user
>> control can be best implemented, and where user intent is known much
>> better.
> The UI has to be agnostic.  So the smarts can't be in the UI.  The API
> can be extended to provide the extra smarts that the UI might need, of
> course.  E.g. we could add to the API a function that sorts/groups the
> entries, so the etags backend can sort them based on "likelyhood" rather
> than group them by file.
>> I very much hope Emacs will continue to be able to support the kind of
>> activities I described above, which AFAIK are very important part of a
>> software developer's job throughout the industry.
> You fail to understand why complaint about etags.el.  I'm not
> complaining about `etags', but about the etags.el front-end, which is in
> need of improvement to handle the case where the user is navigating
> several completely different projects and doesn't want one to pollute
> the other one.
>>> I've tried several times to make real use of it, but always found it
>>> completely unpalatable.  What with having to build those damn TAGS
>>> files, remember to refresh them, remember where they are, constantly
>>> tell Emacs again where they are, deal with its inability to find the
>>> right spot and having to repeat the "C-u M-." finger gymnastics
>>> umpteen times.
>> Those are exaggerations.
> Partly, yes.  I'm just venting my frustration with the tool, and
> pointing out that if xref/elisp (and xref/etags) has some downsides,
> etags.el had its own set of downsides.  And some of those shouldn't be
> that hard to fix (tho they would probably be a lot easier if we didn't
> have to worry about annoying some old-time users because they'd have to
> slightly change their habits).
>> Building TAGS is almost instant, even in Emacs,
> The problem is not computer-time but human-time.
>> you need only refresh them very seldom, and Emacs offers the
>> place from which to load them so you don't need to remember.
> If Emacs knows where the file is, the user shouldn't need to be queried.
>> But this, again, is immaterial for this discussion.  I hope you will
>> agree that, whatever issues we have with etags, replacing it with
>> something that lacks important functionality is not a good idea.
> As I said, going back to etags.el is not an option.
>> That "little detail" all but invalidates most of my use cases.
> Then don't use that backend.  E.g. use xref-etags-mode.
>>> C-u M-. also lets you do loose matching, via completion, if your memory
>>> is failing you.
>> I don't think completion is the right tool for these searches, because
>> the name alone doesn't tell enough.
> Don't pretend you don't know about xref-apropos.
>> See above: you assume that the division of functionality between the
>> UI and the back-ends is at the right place.
> No I don't.  I don't assume the API is fixed either.  All I assume is
> that the UI can't know about the programming language or about the
> quality of any given answer, or any such thing.
>>> But to me, "find-tag" and "find-tag-tag" are not two different
>>> matches to my request.  They're just two completely unrelated
>>> things: either I'm looking for one or I'm looking for the other.
>> Assuming you know what you are looking for, yes.  I described a
>> situation that is frequent for me where you generally don't, at least
>> not well enough to be satisfied by a single exact match.
> And that's not what M-. for.  For that we have xref-apropos.
>> And therein lies its weakness.  I actually don't understand how this
>> kind of assumption could be allowed to exist, when the _default_
> Because this assumption is known to be obtainable, with help from the
> toolchain (the compiler will generally know with 100% certainty the few
> possible definitions matching a particular use).
>         Stefan

[-- Attachment #2: Type: text/html, Size: 6781 bytes --]

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Date: Sat, 02 May 2015 09:59:47 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Thinking about it: for etags it would make sense to group xrefs by name
> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
> "globally"; xrefs are reordered only within a TAGS file.

What if the different TAGS files actually describe the same project,
like lisp/TAGS and src/TAGS in the Emacs case?

Re: UI inconveniences with M-.

[Fumitaka Tokumitsu]

[-- Attachment #1: Type: text/plain, Size: 6401 bytes --]

こなた 

やなまかわ

  おたかたけ 

ま



—
Sent from Mailbox

On Sat, May 2, 2015 at 5:14 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> Cc: emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sat, 2 May 2015 00:11:28 +0300
>> 
>> > If I wanted to talk about the code, I'd say something like
>> > "this-and-that function does wrong things because of so-and-so".
>> 
>> I think both I and Stefan very much wanted you to look at the actual 
>> code here. It's much easier to make demands on functionality than to 
>> propose a clean and modular design.
> One person can only do so much, given her free time, and can only be
> an expert in so many fields.  When you or Stefan report a problem with
> the display engine, or in some other area where I know enough, I don't
> ask for a design before I start working on it.  In this case, all I
> have to offer is user experience, some requirements for what I
> consider to be a good solution, and some general guidelines for such a
> solution (which I only provided in response to repeated demands to do
> so).  If that is not useful, perhaps we should revise our instructions
> for bug reporting.
> IOW, I was reporting problems in an area where I know very little.  I
> don't think it's fair to demand that I provide, let alone code, a
> solution, as a prerequisite for acting on my report.  I think the job
> of making the feature better in response to bug reports is the job of
> those who worked on the feature to begin with, and thus have intimate
> knowledge of the design and the implementation.
>> > Through my naive-user eyes, filtering 140 hits to provide
>> > just a few is perfectly within the capabilities of the UI, at least in
>> > principle.
>> 
>> That is, of course, possible. But then we'll need to define some 
>> universal language-arnostic metadata that the UI would use to operate.
> I think this kind of universal metadata is very much clear and mostly
> already supported.  After all, Emacs has been doing this kind of jobs
> for a very long time, which is why we have notions like e.g. "symbols",
> which each language defines differently.
>> > It sounds more and more like it could be the fault of the design, and
>> > specifically of how functionality is divided between the back-end and
>> > the UI.  Let me elaborate.
>> >
>> > <...>
>> > To me, this means that separating the back-ends from the UI while
>> > leaving the UI "dumb" is basically unworkable, because such a
>> > separation does not really separate anything: there will still be a
>> > very high degree of coherency and cohesion between the two parts.  Or
>> > maybe there will only be one usable back-end, and the rest will
>> > bit-rot.
>> 
>> Sorry, all I see here is a lot of conjecture.
> Of course it is.  What else did you expect from a bystander who wasn't
> involved in the design and implementation?
> It is up to you to do whatever you want with this conjecture.  Some
> people pay a lot of money to get my conjectures in this and similar
> fields.  You get it for free.
>> The current implementations took not a lot of code, and they work
>> well enough.
> That's not an evidence of the design validity, not IME.  This feature
> is with us for too short time to be able to draw conclusions about its
> design.  At least it "didn't work well enough" for me, which is a sign
> that it isn't perfect.  And you already made quite a few changes based
> on my experience.  I think it might be a good time to step back and
> review those changes, looking for some more fundamental issues that
> might benefit from some redesign.  I don't know what you will find, if
> you do that, but I do know that if you continue to insist that the
> design is perfect, you will never see its flaws, if they exist.
>> By the way, CEDET has had a similar feature for a while (try `M-x 
>> semantic-mode', then `C-c , G' on some function, in a C file). Arguably, 
>> even with a better interface.
>> 
>> Any reason why you haven't been using it?
> Partly because CEDET is too heavyweight for most of my needs, and
> partly because I simply didn't have enough time to learn it.
>> > And I'm not even sure your ideas of how to solve that "little detail"
>> > are workable, because of the potentially adverse consequences of
>> > loading code you don't actually need (or want) to execute.  What if
>> > the code is buggy, or dangerous, or simply does things you don't want
>> > to be done in your Emacs session?
>> 
>> That's a valid concern, but you'd have to read that kind of project from 
>> top to bottom first. Then you can load it and proceed to use the 
>> navigation functions.
> That's impossible.  I'm talking about projects whose line counts are
> in hundreds of thousands, sometimes millions.  You cannot read such
> project from top to bottom, when all you need to do is fix some bug or
> find the reason for some particular behavior: you will never make your
> deadline.  Using the tools I'm talking about is the only way to find
> the _relevant_ places which you do need to read and understand.
> IOW, your methodology would put the cart before the horse, in these
> use cases.
>> > I don't know what that means, and don't know enough about JDEE to talk
>> > about it.  In any case, Java is not a "classic" compiled language, as
>> > a Jave development environment is generally capable of running the
>> > code in an interpreter.
>> 
>> There are several ones for C, example: 
>> http://www.drdobbs.com/cpp/ch-a-cc-interpreter-for-script-computing/184402054
>> 
>> Not that it has any relevance to the xref backend interface.
> Then why bring it up?
>> > See above: you assume that the division of functionality between the
>> > UI and the back-ends is at the right place.  I'm not so sure.  If I'm
>> > right, then when more back-ends come, we will see more problems, not
>> > less.
>> 
>> That's conjecture again.
> And I have some gray heir to back it up with.  I have learned from
> long experience that good design is frequently backed up by intuition
> and "conjecture".  I'm not necessarily saying I'm right in this case,
> but what if I am?  Shouldn't you at least consider that, instead of
> rejecting it flatly as "conjecture" and sticking to the original
> design as if it were a scripture?

[-- Attachment #2: Type: text/html, Size: 7417 bytes --]

Re: UI inconveniences with M-.

[Helmut Eller]

On Sat, May 02 2015, Eli Zaretskii wrote:

>> Thinking about it: for etags it would make sense to group xrefs by name
>> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
>> "globally"; xrefs are reordered only within a TAGS file.
>
> What if the different TAGS files actually describe the same project,
> like lisp/TAGS and src/TAGS in the Emacs case?

Always the same: visit-tags-table-buffer returns the TAGS files in a
certain order; say src/TAGS before lisp/TAGS.  Within a TAGS file
find-tag-tag-order or etags-xref-find-definitions-tag-order is used for
sorting but in the final result xrefs from src/TAGS always stay before
those from lisp/TAGS.

I think that's what find-tag-in-order does too, but I'm not 100% sure
because find-tag-in-order and visit-tags-table-buffer are very
complicated.

Helmut

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: emacs-devel@gnu.org,  monnier@iro.umontreal.ca
> Date: Sat, 02 May 2015 11:09:23 +0200
> 
> On Sat, May 02 2015, Eli Zaretskii wrote:
> 
> >> Thinking about it: for etags it would make sense to group xrefs by name
> >> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
> >> "globally"; xrefs are reordered only within a TAGS file.
> >
> > What if the different TAGS files actually describe the same project,
> > like lisp/TAGS and src/TAGS in the Emacs case?
> 
> Always the same: visit-tags-table-buffer returns the TAGS files in a
> certain order; say src/TAGS before lisp/TAGS.  Within a TAGS file
> find-tag-tag-order or etags-xref-find-definitions-tag-order is used for
> sorting but in the final result xrefs from src/TAGS always stay before
> those from lisp/TAGS.

I was asking whether grouping by TAGS file is what the user might want
in this case, since, for example in Emacs, the distinction between
Lisp primitives implemented in C (and thus mentioned in src/TAGS), and
subroutines written in Lisp, does not necessarily exist.  Would the
users mind that, say, forward-line is separated from forward-paragraph?

I don't know the answer to that question, but I think it is worth
considering.

Re: UI inconveniences with M-.

[Helmut Eller]

On Sat, May 02 2015, Eli Zaretskii wrote:

> Would the
> users mind that, say, forward-line is separated from forward-paragraph?

It seems the me that etags users were very satisfied with the order in
which find-tag returned the results.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 10:10 AM, Eli Zaretskii wrote:

> I get "Symbol's function definition is void: mapcan".

Sorry, fixed now.

> I then manually load cl, and re-invoke the command.  The result now is
> "No known references for: set_cursor_from_row".

It expected you to load the tags tables first, in order to know which 
directories to search in. Now it'll prompt for one otherwise.

> In case it matters, the ID file was produced manually with this
> command in the top-level Emacs directory:
>
>    mkid -o ID .

If you want to see it used, then like I said, it needs to be in the same 
directory as TAGS (not one level above). But without it, we default to 
using Grep.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 09:42 AM, Eli Zaretskii wrote:
> I think it's a good idea to add an ID target to the top-level
> Makefile.in.

If you like. But in the Emacs tree, using Grep seems to be fast enough 
already, and its results don't get out of date.

Speaking of ID generation, by default it includes .elc files, so the 
command would need to be something like

mkid --prune=$(find . -name "*.elc")

> Or does Semantic run that automatically as needed?

I don't think so. In any case, you don't need to enable semantic-mode, 
or EDE, to make the search for references work.

> Yep.  At least mkid is lightning-fast (10 sec on the entire Emacs
> tree), so updating ID is not an annoyance IME.

You still need to remember to do that. And automating it to update 
before each search wouldn't be a good idea (searches take less than a 
second).

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 13:11:58 +0300
> 
> On 05/02/2015 10:10 AM, Eli Zaretskii wrote:
> 
> > I get "Symbol's function definition is void: mapcan".
> 
> Sorry, fixed now.

Thanks.

> > I then manually load cl, and re-invoke the command.  The result now is
> > "No known references for: set_cursor_from_row".
> 
> It expected you to load the tags tables first

It did, so that's not the problem.

> Now it'll prompt for one otherwise.

I don't understand this new prompt, it says:

  In directory: /patch/to/emacs/trunk/src/

What does it ask about?  (I just pressed RET, since the directory
sounds correct.)

> > In case it matters, the ID file was produced manually with this
> > command in the top-level Emacs directory:
> >
> >    mkid -o ID .
> 
> If you want to see it used, then like I said, it needs to be in the same 
> directory as TAGS (not one level above).

I tried both, and got the same results.  So this is not the reason;
there's something else at work here.

> But without it, we default to using Grep.

So why didn't it do that?  Why did I get "no known references"?

[Time passes] Ah, I see the reason: Some code that uses ID Utils
expects the file names in the ID database to be absolute.  So if I
create the ID database like this:

   mkid -o ID /path/to/emacs/src

then the feature starts working.

I think it should support relative names as well, by expanding them
relative to the directory in which the ID database was found.  And in
any case, claiming it found no references instead of saying something
about finding no files whose names were returned by ID Utils is a less
than useful error message, IMO.

Now to the Grep alternative: if I rename the ID database, I get an
error message:

  Searching for program: No such file or directory, sh

(This is on MS-Windows.)  The patch for that is below.

But that doesn't end the saga, because after fixing this, I get "No
known references for: set_cursor_from_row" again, when there's no ID
database.  The reason is that the find/grep command uses the wrong
quoting style; the patch below fixes that as well.

And the saga is not yet over, but now I need help.  The method
semantic-symref-parse-tool-output-one-line returns nil, but I cannot
seem to figure out why.  The buffer which contains Grep hits look fine
to me (except that it includes SGR escape sequences, due to
"--color=always" switch to Grep -- why is that a good idea?), but
where's the code of this method?  I couldn't step into it with Edebug,
for some reason.

Help?....

Here's that patch I promised.  I will push it if no one objects.

diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el
index 981dab8..ecb3279 100644
--- a/lisp/cedet/semantic/symref/grep.el
+++ b/lisp/cedet/semantic/symref/grep.el
@@ -115,7 +115,7 @@ (defun semantic-symref-grep-use-template (rootdir filepattern grepflags greppatt
     ;;(message "New command: %s" cmd)
     cmd))
 
-(defcustom semantic-symref-grep-shell "sh"
+(defcustom semantic-symref-grep-shell shell-file-name
   "The shell command to use for executing find/grep.
 This shell should support pipe redirect syntax."
   :group 'semantic
@@ -140,7 +140,8 @@ (cl-defmethod semantic-symref-perform-search ((tool semantic-symref-tool-grep))
 	 (greppat (cond ((eq (oref tool :searchtype) 'regexp)
 			 (oref tool searchfor))
 			(t
-			 (concat "'\\<" (oref tool searchfor) "\\>'"))))
+			 (shell-quote-argument
+                          (concat "\\<" (oref tool searchfor) "\\>")))))
 	 ;; Misc
 	 (b (get-buffer-create "*Semantic SymRef*"))
 	 (ans nil)
@@ -158,10 +159,12 @@ (cl-defmethod semantic-symref-perform-search ((tool semantic-symref-tool-grep))
 	  (let ((cmd (concat "find " default-directory " -type f " filepattern " -print0 "
 			     "| xargs -0 grep -H " grepflags "-e " greppat)))
 	    ;;(message "Old command: %s" cmd)
-	    (call-process semantic-symref-grep-shell nil b nil "-c" cmd)
+	    (call-process semantic-symref-grep-shell nil b nil
+                          shell-command-switch cmd)
 	    )
 	(let ((cmd (semantic-symref-grep-use-template rootdir filepattern grepflags greppat)))
-	  (call-process semantic-symref-grep-shell nil b nil "-c" cmd))
+	  (call-process semantic-symref-grep-shell nil b nil
+                        shell-command-switch cmd))
 	))
     (setq ans (semantic-symref-parse-tool-output tool b))
     ;; Return the answer

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 13:23:34 +0300
> 
> On 05/02/2015 09:42 AM, Eli Zaretskii wrote:
> > I think it's a good idea to add an ID target to the top-level
> > Makefile.in.
> 
> If you like. But in the Emacs tree, using Grep seems to be fast enough 
> already, and its results don't get out of date.

Did you try in src/ or did you try in lisp/?

gid is much faster in large directories, especially if they are
accessed over NFS or some such.  So it scales better.

> Speaking of ID generation, by default it includes .elc files, so the 
> command would need to be something like
> 
> mkid --prune=$(find . -name "*.elc")

Yes.

> > Yep.  At least mkid is lightning-fast (10 sec on the entire Emacs
> > tree), so updating ID is not an annoyance IME.
> 
> You still need to remember to do that.

If you forget, you will be reminded when you find no matches.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Date: Sat, 02 May 2015 14:16:22 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 19468@debbugs.gnu.org
> 
> And the saga is not yet over, but now I need help.  The method
> semantic-symref-parse-tool-output-one-line returns nil, but I cannot
> seem to figure out why.  The buffer which contains Grep hits look fine
> to me (except that it includes SGR escape sequences, due to
> "--color=always" switch to Grep -- why is that a good idea?), but
> where's the code of this method?  I couldn't step into it with Edebug,
> for some reason.

Found it: that --color=always _is_ the problem, since the SGR escapes
interfere with the search.  How does this work on Posix platforms?
Should we filter out that switch unconditionally?

The patch below incorporates the additional fix for this problem, and
now, finally, I have my references using Grep as well:

diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el
index 981dab8..da1803f 100644
--- a/lisp/cedet/semantic/symref/grep.el
+++ b/lisp/cedet/semantic/symref/grep.el
@@ -105,7 +105,11 @@ (defun semantic-symref-grep-use-template (rootdir filepattern grepflags greppatt
   ;; We have grep-compute-defaults.  Let's use it.
   (grep-compute-defaults)
   (let* ((grep-expand-keywords semantic-symref-grep-expand-keywords)
-	 (cmd (grep-expand-template grep-find-template
+	 (cmd (grep-expand-template (if (memq system-type '(windows-nt ms-dos))
+                                        (replace-regexp-in-string
+                                         "--color=always" "--color=auto"
+                                         grep-find-template t t)
+                                      grep-find-template)
 				    greppattern
 				    filepattern
 				    rootdir)))
@@ -115,7 +119,7 @@ (defun semantic-symref-grep-use-template (rootdir filepattern grepflags greppatt
     ;;(message "New command: %s" cmd)
     cmd))
 
-(defcustom semantic-symref-grep-shell "sh"
+(defcustom semantic-symref-grep-shell shell-file-name
   "The shell command to use for executing find/grep.
 This shell should support pipe redirect syntax."
   :group 'semantic
@@ -140,7 +144,8 @@ (cl-defmethod semantic-symref-perform-search ((tool semantic-symref-tool-grep))
 	 (greppat (cond ((eq (oref tool :searchtype) 'regexp)
 			 (oref tool searchfor))
 			(t
-			 (concat "'\\<" (oref tool searchfor) "\\>'"))))
+			 (shell-quote-argument
+                          (concat "\\<" (oref tool searchfor) "\\>")))))
 	 ;; Misc
 	 (b (get-buffer-create "*Semantic SymRef*"))
 	 (ans nil)
@@ -158,10 +163,12 @@ (cl-defmethod semantic-symref-perform-search ((tool semantic-symref-tool-grep))
 	  (let ((cmd (concat "find " default-directory " -type f " filepattern " -print0 "
 			     "| xargs -0 grep -H " grepflags "-e " greppat)))
 	    ;;(message "Old command: %s" cmd)
-	    (call-process semantic-symref-grep-shell nil b nil "-c" cmd)
+	    (call-process semantic-symref-grep-shell nil b nil
+                          shell-command-switch cmd)
 	    )
 	(let ((cmd (semantic-symref-grep-use-template rootdir filepattern grepflags greppat)))
-	  (call-process semantic-symref-grep-shell nil b nil "-c" cmd))
+	  (call-process semantic-symref-grep-shell nil b nil
+                        shell-command-switch cmd))
 	))
     (setq ans (semantic-symref-parse-tool-output tool b))
     ;; Return the answer

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 02:16 PM, Eli Zaretskii wrote:

> It did, so that's not the problem.

Hmm, it worked for me after that.

> I don't understand this new prompt, it says:
>
>    In directory: /patch/to/emacs/trunk/src/
>
> What does it ask about?  (I just pressed RET, since the directory
> sounds correct.)

"[Search for references] in directory". Should the first three words be 
spelled out as well?

> I tried both, and got the same results.  So this is not the reason;
> there's something else at work here.

I simply run mkid, without arguments, in ~/vc/emacs/src.

> [Time passes] Ah, I see the reason: Some code that uses ID Utils
> expects the file names in the ID database to be absolute.  So if I
> create the ID database like this:
>
>     mkid -o ID /path/to/emacs/src
>
> then the feature starts working.

It works fine for me either way (whether called on /path/..., or .), 
using mkid 4.6, on GNU/Linux.

Could it be a misfeature of the Windows port?

> I think it should support relative names as well, by expanding them
> relative to the directory in which the ID database was found.  And in
> any case, claiming it found no references instead of saying something
> about finding no files whose names were returned by ID Utils is a less
> than useful error message, IMO.

ID Utils is used through several levels of indirection. xref can't be 
aware of it. I guess we could output some additional message in 
xref-collect-references though (patch welcome).

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 02:24 PM, Eli Zaretskii wrote:

> Did you try in src/ or did you try in lisp/?

Both. Grep is pretty fast on my machine (1.5 year old laptop, with Intel 
Core i7 and an SSD drive).

> gid is much faster in large directories, especially if they are
> accessed over NFS or some such.  So it scales better.

I'm sure it is, for large codebases. Just not sure if it has any 
tangible advantages for Emacs (and it has a disadvantage: has to be 
updated manually).

> If you forget, you will be reminded when you find no matches.

I might find 1-2 fewer matches than I should, instead of no matches.

And I'm actually aware of how it works. That won't be true for a random 
user.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 02:45 PM, Eli Zaretskii wrote:

> Found it: that --color=always _is_ the problem, since the SGR escapes
> interfere with the search.  How does this work on Posix platforms?
> Should we filter out that switch unconditionally?

It works okay either way here. But if "--color=always" in 
grep-find-template is a problem on Windows, why is it there in the first 
place?

Does grep-find work okay over there?

> The patch below incorporates the additional fix for this problem, and
> now, finally, I have my references using Grep as well:

Nice. I guess that's one way to encourage contributions to CEDET. :)

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 11:12 AM, Eli Zaretskii wrote:

> One person can only do so much, given her free time, and can only be
> an expert in so many fields.  When you or Stefan report a problem with
> the display engine, or in some other area where I know enough, I don't
> ask for a design before I start working on it.

That's not entirely true. Think of bug#18285, for example.

> In this case, all I
> have to offer is user experience, some requirements for what I
> consider to be a good solution, and some general guidelines for such a
> solution (which I only provided in response to repeated demands to do
> so).  If that is not useful, perhaps we should revise our instructions
> for bug reporting.

I'm sure it's useful in the general case. However, in certain situations 
a good design suggestion would be a better argument towards a change 
than only a feature request. You must be aware of that.

> IOW, I was reporting problems in an area where I know very little.  I
> don't think it's fair to demand that I provide, let alone code, a
> solution, as a prerequisite for acting on my report.

The amount of code that you'd have to look at is relatively small, in 
the current case. Just the xref-find-function docstring in xref.el, and 
the 70 lines at the end of etags.el.

> It is up to you to do whatever you want with this conjecture.  Some
> people pay a lot of money to get my conjectures in this and similar
> fields.  You get it for free.

Either way, this analysis doesn't look actionable enough on my side. Sorry.

> That's not an evidence of the design validity, not IME.  This feature
> is with us for too short time to be able to draw conclusions about its
> design.

Sure. We can't know for sure if it's valid. We could only find out that 
it's invalid, at some point.

> At least it "didn't work well enough" for me, which is a sign
> that it isn't perfect.  And you already made quite a few changes based
> on my experience.

Those changes didn't touch of the basics of the design, though. Just on 
implementations (which were initially not much more than proof-of-concept).

> I think it might be a good time to step back and
> review those changes, looking for some more fundamental issues that
> might benefit from some redesign.

Thanks, but this is, again, a very general kind of recommendation. It's 
not my first time writing code, too.

> Partly because CEDET is too heavyweight for most of my needs, and
> partly because I simply didn't have enough time to learn it.

Okay. But note that when you're asking for features that it already 
provides for some extent (semantic-symref supports renames; did you know 
that?), for us to create a better user experience we'll need members of 
the target audience to try their best to actually use it and report on 
the pain points.

> That's impossible.  I'm talking about projects whose line counts are
> in hundreds of thousands, sometimes millions.  You cannot read such
> project from top to bottom, when all you need to do is fix some bug or
> find the reason for some particular behavior:

If you can't read it whole, you continue to be in danger of malicious 
behavior tucked somewhere in the codebase. Would it really be better to 
leave it until production deployment, instead of allowing to happen on 
the developer's machine?

> IOW, your methodology would put the cart before the horse, in these
> use cases.

In any case, it's a well-known problem, and people have been known to 
live with that choice.

> I'm not necessarily saying I'm right in this case,
> but what if I am?  Shouldn't you at least consider that, instead of
> rejecting it flatly as "conjecture" and sticking to the original
> design as if it were a scripture?

I'm not rejecting. But like you said, one person can only do so much. 
When I don't see a clear path to action that would satisfy both the 
original requirements, as well as the ones you add, the estimated amount 
of work to get to the ideal is unbounded.

There are other features (as well as different projects) I'd like to 
work on.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:01:59 +0300
> 
> > I don't understand this new prompt, it says:
> >
> >    In directory: /patch/to/emacs/trunk/src/
> >
> > What does it ask about?  (I just pressed RET, since the directory
> > sounds correct.)
> 
> "[Search for references] in directory".

Do you mean it will only return matches in files in that directory,
even if ID says the file is elsewhere?

> Should the first three words be spelled out as well?

I cannot answer that without understanding the meaning of the prompt.

> > [Time passes] Ah, I see the reason: Some code that uses ID Utils
> > expects the file names in the ID database to be absolute.  So if I
> > create the ID database like this:
> >
> >     mkid -o ID /path/to/emacs/src
> >
> > then the feature starts working.
> 
> It works fine for me either way (whether called on /path/..., or .), 
> using mkid 4.6, on GNU/Linux.

If you type "lid", do you see absolute file names in both cases, or do
you see relative ones when ID was created by "mkid ." (note the dot)?

> Could it be a misfeature of the Windows port?

If you tell me that "lid" returns relative file names, and
xref-find-references still work, I will see if this is something
specific to Windows.

> > I think it should support relative names as well, by expanding them
> > relative to the directory in which the ID database was found.  And in
> > any case, claiming it found no references instead of saying something
> > about finding no files whose names were returned by ID Utils is a less
> > than useful error message, IMO.
> 
> ID Utils is used through several levels of indirection. xref can't be 
> aware of it.

I didn't say xref should fix this.  Is there no information  xref gets
from symref apart of the (empty) list of references?

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:07:09 +0300
> 
> On 05/02/2015 02:24 PM, Eli Zaretskii wrote:
> 
> > Did you try in src/ or did you try in lisp/?
> 
> Both. Grep is pretty fast on my machine (1.5 year old laptop, with Intel 
> Core i7 and an SSD drive).

Well, with SSD drive, what do you want?.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:16:24 +0300
> 
> On 05/02/2015 02:45 PM, Eli Zaretskii wrote:
> 
> > Found it: that --color=always _is_ the problem, since the SGR escapes
> > interfere with the search.  How does this work on Posix platforms?
> > Should we filter out that switch unconditionally?
> 
> It works okay either way here.

So how do you feel about filtering it out it on all platforms, not
just on Windows?

> But if "--color=always" in grep-find-template is a problem on
> Windows, why is it there in the first place?

I guess because grep-find converts it into colored display using
ansi-color or some such.

> Does grep-find work okay over there?

Yes, of course.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:41:22 +0300
> 
> On 05/02/2015 11:12 AM, Eli Zaretskii wrote:
> 
> > One person can only do so much, given her free time, and can only be
> > an expert in so many fields.  When you or Stefan report a problem with
> > the display engine, or in some other area where I know enough, I don't
> > ask for a design before I start working on it.
> 
> That's not entirely true. Think of bug#18285, for example.

Which part of what I said was not entirely true there?

> > In this case, all I
> > have to offer is user experience, some requirements for what I
> > consider to be a good solution, and some general guidelines for such a
> > solution (which I only provided in response to repeated demands to do
> > so).  If that is not useful, perhaps we should revise our instructions
> > for bug reporting.
> 
> I'm sure it's useful in the general case. However, in certain situations 
> a good design suggestion would be a better argument towards a change 
> than only a feature request. You must be aware of that.

If I had a good design suggestion, I'd certainly not withhold it.

> > IOW, I was reporting problems in an area where I know very little.  I
> > don't think it's fair to demand that I provide, let alone code, a
> > solution, as a prerequisite for acting on my report.
> 
> The amount of code that you'd have to look at is relatively small, in 
> the current case. Just the xref-find-function docstring in xref.el, and 
> the 70 lines at the end of etags.el.

xref.el uses facilities I never used, so for me it means I'd need to
study those as well.  Please believe me that when I said it's more
than I could invest, I already explored some of the code.

> > Partly because CEDET is too heavyweight for most of my needs, and
> > partly because I simply didn't have enough time to learn it.
> 
> Okay. But note that when you're asking for features that it already 
> provides for some extent (semantic-symref supports renames; did you know 
> that?), for us to create a better user experience we'll need members of 
> the target audience to try their best to actually use it and report on 
> the pain points.

I can be that user, at least in some cases.  Although the experience
of this bug report doesn't encourage me to get involved in more.

> > That's impossible.  I'm talking about projects whose line counts are
> > in hundreds of thousands, sometimes millions.  You cannot read such
> > project from top to bottom, when all you need to do is fix some bug or
> > find the reason for some particular behavior:
> 
> If you can't read it whole, you continue to be in danger of malicious 
> behavior tucked somewhere in the codebase. Would it really be better to 
> leave it until production deployment, instead of allowing to happen on 
> the developer's machine?

I'm talking about projects that are already deployed, sometimes for
years.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 03:49 PM, Eli Zaretskii wrote:

> So how do you feel about filtering it out it on all platforms, not
> just on Windows?

I don't see a downside, so far. If any bug reports arise from it, I know 
who to forward them to. :)

xref "find references" and grouping

[Dmitry Gutov]

On 05/02/2015 09:59 AM, Stefan Monnier wrote:

> I don't see much need for nesting.  I think the issue is mostly one of
> sorting in *xref* (or in the cycling, if we were to use a hypothetical
> other UI that provides functionality like the old etags.el).

Take a look at M-x semantic-symref output, and compare to what we have 
with M-x xref-find-references. Here's a screenshot of the former:

http://cedet.sourceforge.net/img-gen/symref.png

Note the grouping by the "current function", as well as by file name. It 
seems to me that it can be pretty valuable.

Here's how a similar interface looks in a popular proprietary IDE: 
https://www.jetbrains.com/img/webhelp/idea/findUsageField.png

Helmut, any suggestions? Does SLIME have anything related?

> So, let's focus on this.
> My suggestion would be either:
> - make xref-find-function return not just a list of locations, but also
>    an indication whether they're pre-sorted or not.
> - make xref-find-function return not a list of locations, but a list of
>    lists of locations (i.e. every location inside a sub-list is
>    considered to have the same "level/quality/likelihood", but the lists
>    are sorted by their "level/quality/likelihood").
> - let the backend provide its own sorting function to override the
>    current "group by file" sorting.

The question of sorting would need resolving indeed.

> Basically, I think the focus should be on making sure that we can
> re-implement the old behavior on top of the new API when using the
> xref/etags backend.

We also can't group xref-find-apropos matches by likelihood, I think. Or 
would you include file path on each line? That seems wasteful.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 03:57 PM, Eli Zaretskii wrote:

>> That's not entirely true. Think of bug#18285, for example.
>
> Which part of what I said was not entirely true there?

We've discussed the design, to some extent? And the tradeoffs?

At least that's how it looked for me.

> xref.el uses facilities I never used, so for me it means I'd need to
> study those as well.  Please believe me that when I said it's more
> than I could invest, I already explored some of the code.

I specifically mentioned only looking a xref-find-function in xref.el, 
not anything else. That's the whole point of the abstraction.

But I've no reason to disbelieve you anyway.

> I can be that user, at least in some cases.  Although the experience
> of this bug report doesn't encourage me to get involved in more.

You've got the "find references" implementation out of it, haven't you?

> I'm talking about projects that are already deployed, sometimes for
> years.

And yet you're worried what one of them could do to your machine when 
loaded into memory?

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:01:59 +0300
> 
> > I tried both, and got the same results.  So this is not the reason;
> > there's something else at work here.
> 
> I simply run mkid, without arguments, in ~/vc/emacs/src.
> 
> > [Time passes] Ah, I see the reason: Some code that uses ID Utils
> > expects the file names in the ID database to be absolute.  So if I
> > create the ID database like this:
> >
> >     mkid -o ID /path/to/emacs/src
> >
> > then the feature starts working.
> 
> It works fine for me either way (whether called on /path/..., or .), 
> using mkid 4.6, on GNU/Linux.
> 
> Could it be a misfeature of the Windows port?

Found the reason, the patch is below.

There's some tricky logic in ID Utils wrt when to report relative file
names and when not, and I guess CEDET doesn't expect absolute file
names, especially not on Windows.  With the patch below, it works for
me both ways.

diff --git a/lisp/cedet/semantic/symref/idutils.el b/lisp/cedet/semantic/symref/idutils.el
index c22a6a3..655b000 100644
--- a/lisp/cedet/semantic/symref/idutils.el
+++ b/lisp/cedet/semantic/symref/idutils.el
@@ -60,7 +60,7 @@ (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-
 	 (when (re-search-forward "^\\([^ ]+\\) " nil t)
 	   (match-string 1)))
 	(t
-	 (when (re-search-forward "^\\([^ :]+\\):+\\([0-9]+\\):" nil t)
+	 (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t)
 	   (cons (string-to-number (match-string 2))
 		 (expand-file-name (match-string 1) default-directory))
 	   ))))

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 03:45 PM, Eli Zaretskii wrote:

> Do you mean it will only return matches in files in that directory,
> even if ID says the file is elsewhere?

I don't know. Can ID file reference files outside of its subtree?

One would also need to study GNU Global and CScope in that context.

> I cannot answer that without understanding the meaning of the prompt.

But you asked a good question.

> If you type "lid", do you see absolute file names in both cases, or do
> you see relative ones when ID was created by "mkid ." (note the dot)?

Relative ones, in both cases. You might want to look at 
`cedet-idutils-expand-filename'.

> I didn't say xref should fix this.  Is there no information  xref gets
> from symref apart of the (empty) list of references?

Not currently, no.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 16:31:13 +0300
> 
> On 05/02/2015 03:57 PM, Eli Zaretskii wrote:
> 
> >> That's not entirely true. Think of bug#18285, for example.
> >
> > Which part of what I said was not entirely true there?
> 
> We've discussed the design, to some extent? And the tradeoffs?

Yes, but I didn't demand that.

> > I can be that user, at least in some cases.  Although the experience
> > of this bug report doesn't encourage me to get involved in more.
> 
> You've got the "find references" implementation out of it, haven't you?

Yes, thanks.  I also got shouted at.

> > I'm talking about projects that are already deployed, sometimes for
> > years.
> 
> And yet you're worried what one of them could do to your machine when 
> loaded into memory?

Yes.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 16:42:28 +0300
> 
> On 05/02/2015 03:45 PM, Eli Zaretskii wrote:
> 
> > Do you mean it will only return matches in files in that directory,
> > even if ID says the file is elsewhere?
> 
> I don't know. Can ID file reference files outside of its subtree?

It can.  But if the subdirectories of the directory it asks about will
also be included, that's good enough.  Perhaps the prompt should say

  Search in and below directory: ...

> > If you type "lid", do you see absolute file names in both cases, or do
> > you see relative ones when ID was created by "mkid ." (note the dot)?
> 
> Relative ones, in both cases. You might want to look at 
> `cedet-idutils-expand-filename'.

I don't think it's relevant, see my other message.

> > I didn't say xref should fix this.  Is there no information  xref gets
> > from symref apart of the (empty) list of references?
> 
> Not currently, no.

Maybe the interface could be expanded to provide more in case of an
error.

Re: xref "find references" and grouping

[Helmut Eller]

On Sat, May 02 2015, Dmitry Gutov wrote:

> Helmut, any suggestions?

Maybe the current group label could be generalized to a path in a tree
e.g. (<file> <class> <method>) where each element of such a tree path is
simply a string and the UI can display it either as a flat list or some
of kind tree widget.

> Does SLIME have anything related?

Not really.  For a time we grouped compiler error/warning/notes with a
kind of tree widget; the idea was to make it easier to skip over notes
and warnings and see error more quickly, but in practice much time was
spend on opening and closing those subtrees.  I don't know if a tree
widget is a win.  In Emacs such things always seem to require a lot of
extra screen space compared to the GUI widgets in other IDEs.

Helmut

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 04:41 PM, Eli Zaretskii wrote:
> There's some tricky logic in ID Utils wrt when to report relative file
> names and when not, and I guess CEDET doesn't expect absolute file
> names, especially not on Windows.  With the patch below, it works for
> me both ways.

It doesn't break anything here, please go ahead.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 05:13 PM, Eli Zaretskii wrote:

> It can.  But if the subdirectories of the directory it asks about will
> also be included, that's good enough.  Perhaps the prompt should say

"If". Continuing that line of thought, wouldn't it be possible that the 
ID file contains only files from outside of its subtree in its index? 
Then the prompt will be entirely wrong.

>    Search in and below directory: ...

When I say (or read) "search in directory", I understand it to mean the 
subdirectories, too.

> Maybe the interface could be expanded to provide more in case of an
> error.

But why would that be an error? "Nothing found" is an entirely 
legitimate result.

Since we've encountered a bug in the code, maybe some extra assertions 
in `semantic-symref-parse-tool-output-one-line' (or in its caller?) 
would take care of that.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 04:44 PM, Eli Zaretskii wrote:

> Yes, but I didn't demand that.

There's no demand, I'd say. But the discussion has stopped making real 
progress without it.

The aforementioned bug is also at standstill, you might notice.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 10:59 AM, Helmut Eller wrote:

> Theoretically this can already be done by ignoring xref-location-group
> or by specializing xref-location-group for xref-etags-location so that it
> returns always the same group.

Or, if we try to be extra-informative, group by find-tag-tag-order 
functions (maybe replacing each name with some words in human language; 
or classify them into several groups, too). But we won't have any place 
to write file names to.

> Thinking about it: for etags it would make sense to group xrefs by name
> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
> "globally"; xrefs are reordered only within a TAGS file.

Indeed. But when combined with the above, it will be 2-3 levels of grouping.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 20:41:59 +0300
> 
> On 05/02/2015 05:13 PM, Eli Zaretskii wrote:
> 
> > It can.  But if the subdirectories of the directory it asks about will
> > also be included, that's good enough.  Perhaps the prompt should say
> 
> "If". Continuing that line of thought, wouldn't it be possible that the 
> ID file contains only files from outside of its subtree in its index? 
> Then the prompt will be entirely wrong.

We could always ask about the ID file itself, you know...

> >    Search in and below directory: ...
> 
> When I say (or read) "search in directory", I understand it to mean the 
> subdirectories, too.

Well, it wasn't entirely clear, perhaps because I know too much about
ID Utils.

> > Maybe the interface could be expanded to provide more in case of an
> > error.
> 
> But why would that be an error? "Nothing found" is an entirely 
> legitimate result.

That's not what happened here: the invocation of 'lid' yields a
non-empty buffer, but then the result is an empty list.  That's a
clear sign of some problem somewhere.

> Since we've encountered a bug in the code, maybe some extra assertions 
> in `semantic-symref-parse-tool-output-one-line' (or in its caller?) 
> would take care of that.

Could be.  In general, if the buffer is non-empty, there should be
results.

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/02/2015 05:14 PM, Helmut Eller wrote:

> Maybe the current group label could be generalized to a path in a tree
> e.g. (<file> <class> <method>) where each element of such a tree path is
> simply a string and the UI can display it either as a flat list or some
> of kind tree widget.

In this format, we lose the ordering, though. Then we'll have sorting 
functions, which would need to be returned with the data, and we'll need 
to decide whether there's one function, or several (per level).

Or we could tackle this in OO fashion.

The different actions in xref-find-function will be required to return 
"renderable" elements (better name pending), in a backend-defined order, 
and there will be no subsequent sorting (basically, like now).

Each of those will implement a set of generic methods:

- Render itself in the xref buffer.
- Return the xref location element at point.
- Move point to the previous or next location element.

Later, some extra stuff, if we reach the "rename" feature, like:

- Return all locations inside.
- Mark or unmark all locations inside.

(Marked location elements would render themselves differently).

The downsides, as I see them:

- Less transparent data structures than a flat list.
- Some overhead on virtual dispatch (but hopefully it'll be negligible).
- Backends having a lot more say in how the buffer is displayed, and 
thus having more opportunities to break the rendering, use their own 
extra-special faces, and reinvent the interface (for better or worse).

The last point might also yield benefits: the generally agreed upon UI 
improvements could then be pushed into the default rendering logic.

> Not really.  For a time we grouped compiler error/warning/notes with a
> kind of tree widget; the idea was to make it easier to skip over notes
> and warnings and see error more quickly, but in practice much time was
> spend on opening and closing those subtrees.  I don't know if a tree
> widget is a win.  In Emacs such things always seem to require a lot of
> extra screen space compared to the GUI widgets in other IDEs.

Please note that folding (and extra indentation for sub-elements) is not 
a necessary element of what I'm describing. We can just use different 
faces for group headings, and otherwise render everything flat (if 
anyone disagrees, it's easy to make customizable).

Most of the time I dislike code folding, and the idea to conflate 
expanding of the matches with marking (for renaming) in semantic-symref 
strikes me as unfortunate. It still looks nice, though.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 09:59 AM, Stefan Monnier wrote:

> Another thing that we should consider is how to let the user choose
> which backend to use.  Currently we only have xref-etags-mode, but
> I think it would make sense to make it possible for the user to
> dynamically choose among the potentially many more backends (e.g. we
> could add an xref/grep backend, and there will hopefulyl be more using
> things like id-utils, gtags, ...)

I still expect that managing those simply with minor modes will work 
well enough. While using id-utils or gtags instead of etags can be 
valuable, we'd probably stick to only one of them at a time.

Maybe choose between them automatically, like semantic-symref does (and 
allows us to do).

> and maybe also to use several backends
> at the same time (merging the results).

If we agree on the OO approach to grouping (or even the list-of-strings 
approach), merging them automatically would look pretty unsolvable to 
me. Concatenation would work, though (with the OO approach).

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 09:45 PM, Eli Zaretskii wrote:

> We could always ask about the ID file itself, you know...

At that point, we don't yet know if ID Utils will be used, or GNU 
Global, or something else.

Maybe we should forget this complication and simply ask the user to 
visit the tags file first (it's the etags backend, after all).

That just seemed awkward to me, because the contents of that file aren't 
used at all in this scenario (only its location).

>> When I say (or read) "search in directory", I understand it to mean the
>> subdirectories, too.
>
> Well, it wasn't entirely clear, perhaps because I know too much about
> ID Utils.

How come that's a factor? 'mkid' scans the subdirectories.

> That's not what happened here: the invocation of 'lid' yields a
> non-empty buffer, but then the result is an empty list.  That's a
> clear sign of some problem somewhere.

Yup.

> Could be.  In general, if the buffer is non-empty, there should be
> results.

Not a bad heuristic. The buffer said "No 'ID' file found" when I was 
dealing with a bug in my code. But that wass an error as well.

I think we can add that non-empty assertion to 
`semantic-symref-parse-tool-output'.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 22:17:22 +0300
> 
> >> When I say (or read) "search in directory", I understand it to mean the
> >> subdirectories, too.
> >
> > Well, it wasn't entirely clear, perhaps because I know too much about
> > ID Utils.
> 
> How come that's a factor? 'mkid' scans the subdirectories.

Exactly.  So it was weird to be asked about a directory.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 10:45 PM, Eli Zaretskii wrote:

> Exactly.  So it was weird to be asked about a directory.

It's the directory we expect the ID file to be in. Or GPATH, or 
cscope.out. Or none of them (then we use Grep).

Maybe you expect the default directory to be used automatically, but I 
wouldn't like to optimize for "flat" code bases.

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/30/2015 08:46 PM, Helmut Eller wrote:

> next-error calls next-error-find-buffer.  Maybe we could record every
> buffer that was returned by next-error-find-buffer and include them in
> the candidate list.

That's not terrible, but then we have almost the same two non-ideal choices:

- Only record the buffers that have even been noticed as the value of 
next-error-last-buffer. This way we'll ignore e.g. Compilation buffers 
that have never had a chance to be used with `next-error'.

- Record all buffers that ever were returned from that function. This 
will also include normal (file-visiting) buffers where 
next-error-function is set locally (and usually only points to locations 
within that buffer).

Here's a different heuristic: return all buffers where 
next-error-function is set, but buffer-file-name is nil. Probably not 
ideal, but it'll clearly delineate between normal and special buffers.

I guess that will only leave inability, at certain times, to choose the 
current buffer's next-error-function over some Compilation buffer's one 
(http://debbugs.gnu.org/20489).

Re: conflicting uses of next-error-function

[Helmut Eller]

On Sun, May 03 2015, Dmitry Gutov wrote:

> On 04/30/2015 08:46 PM, Helmut Eller wrote:
>
>> next-error calls next-error-find-buffer.  Maybe we could record every
>> buffer that was returned by next-error-find-buffer and include them in
>> the candidate list.
>
> That's not terrible, but then we have almost the same two non-ideal choices:
>
> - Only record the buffers that have even been noticed as the value of
> next-error-last-buffer. This way we'll ignore e.g. Compilation buffers
> that have never had a chance to be used with `next-error'.
>
> - Record all buffers that ever were returned from that function. This
> will also include normal (file-visiting) buffers where
> next-error-function is set locally (and usually only points to
> locations within that buffer).

I was thinking of adding a new function like

 (next-error-note-buffer BUFFER &optional DONT-SELECT)

that essentially replaces next-error-last-buffer for those packages that
bother to use it AND to add some degree of backward compatibility by
recording the buffer returned by next-error-find-buffer.

Helmut

Re: xref "find references" and grouping

[Helmut Eller]

On Sat, May 02 2015, Dmitry Gutov wrote:

> On 05/02/2015 05:14 PM, Helmut Eller wrote:
>
>> Maybe the current group label could be generalized to a path in a tree
>> e.g. (<file> <class> <method>) where each element of such a tree path is
>> simply a string and the UI can display it either as a flat list or some
>> of kind tree widget.
>
> In this format, we lose the ordering, though. Then we'll have sorting
> functions, which would need to be returned with the data, and we'll
> need to decide whether there's one function, or several (per level).

Not sure what you mean.  Trees can be ordered and a list of tree-paths
would seem enough to describe an ordered tree.

[...]
> The different actions in xref-find-function will be required to return
> "renderable" elements (better name pending), in a backend-defined
> order, and there will be no subsequent sorting (basically, like now).

Sounds like there would not be much left to be shared between different
languages.  Maybe we should never have tried to generalize this anyway
and simply declare a convention that programming modes should bind
M-. to a find-definition-like command.  That certainly would avoid the
current dilemma of pleasing everybody.

[...]
> Later, some extra stuff, if we reach the "rename" feature, like:

A rename feature would probably benefit from an UI that's
separate/independent from find-definition.

Helmut

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/03/2015 09:54 AM, Helmut Eller wrote:

> I was thinking of adding a new function like
>
>   (next-error-note-buffer BUFFER &optional DONT-SELECT)
>
> that essentially replaces next-error-last-buffer for those packages that
> bother to use it AND to add some degree of backward compatibility by
> recording the buffer returned by next-error-find-buffer.

I'm not sure I understand your proposal. You seem to have ignored the 
description of the "non-ideal choice" number two.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Sun, May 03 2015, Dmitry Gutov wrote:

> On 05/03/2015 09:54 AM, Helmut Eller wrote:
>
>> I was thinking of adding a new function like
>>
>>   (next-error-note-buffer BUFFER &optional DONT-SELECT)
>>
>> that essentially replaces next-error-last-buffer for those packages that
>> bother to use it AND to add some degree of backward compatibility by
>> recording the buffer returned by next-error-find-buffer.
>
> I'm not sure I understand your proposal. You seem to have ignored the
> description of the "non-ideal choice" number two.

Sorry, hadn't fully understood option two.  Yes, it option two probably
covers most cases that are relevant in practice.

Helmut

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/03/2015 04:25 PM, Helmut Eller wrote:

> Sorry, hadn't fully understood option two.  Yes, it option two probably
> covers most cases that are relevant in practice.

Here's the problem: with global-flycheck-mode on, all emacs-lisp-mode 
buffers have next-error-function set locally (to 
flycheck-next-error-function). And that function navigates between 
byte-compilation errors and warnings within the current buffer, which is 
very much in line with the next-error-function docstring (which says 
"find the next error in the current buffer").

Thus quite often next-error-find-buffer will return any emacs-lisp-mode 
buffer (for instance, when it's the only one visible in the current 
frame). If we remember all such buffers,

a) In a long-running Emacs session we'll remember a lot of them.

b) Setting next-error-last-buffer, while in one emacs-lisp-mode buffer, 
to another emacs-lisp-mode buffer doesn't make sense, because the latter 
only contains local error positions.

Thus, the resulting remembered list will be ill-suited for completion in 
the "change next-error-last-buffer" command.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:16:24 +0300
> 
> On 05/02/2015 02:45 PM, Eli Zaretskii wrote:
> 
> > Found it: that --color=always _is_ the problem, since the SGR escapes
> > interfere with the search.  How does this work on Posix platforms?
> > Should we filter out that switch unconditionally?
> 
> It works okay either way here. But if "--color=always" in 
> grep-find-template is a problem on Windows, why is it there in the first 
> place?

I recalled the reason: grep-find wants to display the colorification,
by converting SGR escapes into Emacs faces, so it needs those
escapes.  However, since on Windows subprocesses are run via pipes,
which fail the isatty test, Grep will not produce colors under "auto",
so we use "always" instead.

CEDET shouldn't have blindly copied the template, because its needs
are different, and in particular it doesn't need or want the colors
anyway.

Re: xref "find references" and grouping

[Stefan Monnier]

> languages.  Maybe we should never have tried to generalize this anyway
> and simply declare a convention that programming modes should bind
> M-. to a find-definition-like command.  That certainly would avoid the
> current dilemma of pleasing everybody.

Actually, the current tension doesn't come from xref.el but comes from
the fact that we now use find-func.el instead of etags.el by default in
Elisp mode buffers.  The use of xref.el should actually let use solve
this dilemma, rather than introduce it.


        Stefan

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/03/2015 10:47 AM, Helmut Eller wrote:

> Not sure what you mean.  Trees can be ordered and a list of tree-paths
> would seem enough to describe an ordered tree.

Ok, you're right. There's nothing impossible in a comparison function 
accepting lists of different lengths.

I can offer another, recently surfaced critique: if all groups are 
simply strings, we can't do content-specific things to them at display. 
For instance, can't shorten file paths.

Tree widgets can have specialized subclasses.

> Sounds like there would not be much left to be shared between different
> languages.

That's a very surprising conclusion. What prompted it? Like you said, 
we're not sorting now anyway. There's not much language-specific in the 
proposal.

A set of widgets (like generic group, file group, method group, and a 
few kinds of locations) will allow any language mode to return a 
structured set of matches, which can even be rendered in different ways.

> Maybe we should never have tried to generalize this anyway
> and simply declare a convention that programming modes should bind
> M-. to a find-definition-like command.  That certainly would avoid the
> current dilemma of pleasing everybody.

By allowing every mode to behave in its own way? That would certainly 
please every user.

> A rename feature would probably benefit from an UI that's
> separate/independent from find-definition.

It would be different, but not necessarily vastly different. And the set 
of returned xrefs we're getting from the "find references" command is 
already enough to base a simplistic rename on.

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Dmitry Gutov on Mon, 4 May 2015 05:09:17 +0300 wrote:

 > A set of widgets (like generic group, file group, method group, and a few kinds
 > of locations) will allow any language mode to return a structured set of
 > matches, which can even be rendered in different ways.

Ho about a simple tabular interface. Each back-end will return a list of
matches with a backend specific plist of metadata. For example:

  (foo . (:file "path/to/foo-file" :line 23 :priority 1 :type "method" :class "baz"))

Then arrange the xref in a simple flat table:

   File                  Priority  Type    Class  Target
   path/to/foo-file:23   1         method  bax    foo
   ...                   ...       ...     ...    ...

Then add a sorting command to sort on File, Priority Type etc. and
introduced a custom var for the default sorting.

There is a mode in emacs for tabular display, I forgot its name but I
think it already takes care of the sorting and other useful stuff.

Note that sorting with trees is not so straightforward. Depending on
sorting criteria you will get a very different trees in terms of inner
structure.

Trees also tend to waste a lot of space and are more demanding
cognitively because you need to figure out where one tree starts and
where it ends. As the "height" of each tree is different that could be a
rather hard task unless you use a clear delimiter between trees. But
that means using additional vertical space for delimiters.

One huge advantage of flat displays is that you can narrow candidates on
whatever component. So you can simply use selective display like occur,
helm-occur or ivy to narrow your buffer. You cannot do that with trees.

 Vitalie

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Stefan Monnier on Sat, 02 May 2015 02:59:28 -0400 wrote:

 > - make xref-find-function return not a list of locations, but a list of
 >   lists of locations (i.e. every location inside a sub-list is
 >   considered to have the same "level/quality/likelihood", but the lists
 >   are sorted by their "level/quality/likelihood").

That would allow for one hard-codded sorting. I think it's much better
to return a plist of metadata for each candidates. Then you can order on
multiple criteria.

 > - let the backend provide its own sorting function to override the
 >   current "group by file" sorting.

I think that's the task for the user to decide. The backend can compute
it's own :priority meta score and pass it back. If user's custom var
says sort on :priority then sort on priority.

Some backends might return more meta than others, so user's option
should be a list (:priority :type :file). The sorting algorithm should
got through the list and sort on first component which is defined. :file
must be defined for all backends.

 > I think it would make sense to make it possible for the user to
 > dynamically choose among the potentially many more backends (e.g. we
 > could add an xref/grep backend, and there will hopefulyl be more using
 > things like id-utils, gtags, ...) and maybe also to use several backends
 > at the same time (merging the results).  

Ho about the following UI. Instead of one set of backends you have
multiple levels of increasing sets of backends.

At each moment of time there is only one active "set" which you can
change with `xref-rotate-level`. The user then can customize levels of
backends:

  xref-backend-level-1 . (xref-default-mode-backend)
  xref-backend-level-2 . (xref-etags elisp-xref)
  xref-backend-level-3 . ....


A set corresponding to level N would automatically include all backends
from the lower levels. In the above example, if you set the level to 2
xref will merge default-backend, etags and elisp-xref.

Thus the UI is only 2 commands:

   xref-add-backend-in-level (ask for level and on prefix remove backend)
   xref-rotate-level (with numeric prefix, set level)

Users can then customize each level per mode or globally and thus have
consistent levels that they can rely upon independently of the context.

Vitalie

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Vitalie Spinu on Mon, 04 May 2015 15:10:35 +0200 wrote:
 > There is a mode in emacs for tabular display, I forgot its name but I
 > think it already takes care of the sorting and other useful stuff.

It's tabulated-list.el; same thing that package.el is based upon.

  Vitalie

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> Cc: 19468@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 20:35:56 +0300
> 
> On 05/02/2015 04:41 PM, Eli Zaretskii wrote:
> > There's some tricky logic in ID Utils wrt when to report relative file
> > names and when not, and I guess CEDET doesn't expect absolute file
> > names, especially not on Windows.  With the patch below, it works for
> > me both ways.
> 
> It doesn't break anything here, please go ahead.

Done, with a couple more such blunders.  My favorite is this (from
cedet/semantic/bovine/gcc.el):

                (when (file-accessible-directory-p path)
                  (when (if (memq system-type '(windows-nt))
                            (/= ?/ (nth 1 chars))  <<<<<<<<<<<<<<<<
                          (= ?/ (nth 1 chars)))    <<<<<<<<<<<<<<<<
                    (add-to-list 'inc-path
                                 (expand-file-name (substring line 1))

which evidently wants to make sure the file name is in absolute form.

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/04/2015 04:10 PM, Vitalie Spinu wrote:

> Note that sorting with trees is not so straightforward. Depending on
> sorting criteria you will get a very different trees in terms of inner
> structure.

You can still have a sorting command, and sort the output when it's a 
tree. The process would admittedly less obvious than with a tabulated 
list widget, though.

> Trees also tend to waste a lot of space and are more demanding

Again, we heavily disagree about what a waste of space is. In my mind, 
repeating the same value certainly qualifies.

> cognitively because you need to figure out where one tree starts and
> where it ends. As the "height" of each tree is different that could be a
> rather hard task unless you use a clear delimiter between trees. But
> that means using additional vertical space for delimiters.

There's no need for delimiters, I think. Just use different faces for 
group headers, and hope that tree branches of different lengths won't 
happen too often.

> One huge advantage of flat displays is that you can narrow candidates on
> whatever component. So you can simply use selective display like occur,
> helm-occur or ivy to narrow your buffer. You cannot do that with trees.

That's an advantage, yes. However, how are you even going to fit the 
xref-find-references output in a tabulated list? It contains full lines. 
If you put them in a column, the rest of the columns will surely overflow.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/04/2015 04:41 PM, Vitalie Spinu wrote:

> At each moment of time there is only one active "set" which you can
> change with `xref-rotate-level`. The user then can customize levels of
> backends:
>
>    xref-backend-level-1 . (xref-default-mode-backend)
>    xref-backend-level-2 . (xref-etags elisp-xref)
>    xref-backend-level-3 . ....
>
>
> A set corresponding to level N would automatically include all backends
> from the lower levels. In the above example, if you set the level to 2
> xref will merge default-backend, etags and elisp-xref.

I'd rather not concern myself with this kind of complexity. However, if 
you're fine with merge logic simply concatenating the results (and it 
will be hard to do better), this shouldn't be hard to implement on top 
of the basic one-backend-at-a-time xref system, in a minor mode.

Re: conflicting uses of next-error-function

[Ted Zlatanov]

On Sun, 3 May 2015 17:12:42 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> Here's the problem: with global-flycheck-mode on, all emacs-lisp-mode
DG> buffers have next-error-function set locally (to
DG> flycheck-next-error-function). And that function navigates between
DG> byte-compilation errors and warnings within the current buffer, which
DG> is very much in line with the next-error-function docstring (which
DG> says "find the next error in the current buffer").

DG> Thus quite often next-error-find-buffer will return any
DG> emacs-lisp-mode buffer (for instance, when it's the only one visible
DG> in the current frame). If we remember all such buffers,

If we agree to implement a `next-error-priority', then perhaps
`next-error-find-buffer' should penalize buried buffers by subtracting
from their priority.  Thus buffers most recently buried will have the
lowest priority.

Ted

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 00:29:02 +0300 wrote:

 >> One huge advantage of flat displays is that you can narrow candidates on
 >> whatever component. So you can simply use selective display like occur,
 >> helm-occur or ivy to narrow your buffer. You cannot do that with trees.
 
 > That's an advantage, yes. However, how are you even going to fit the
 > xref-find-references output in a tabulated list?

Just make it the last column and abbreviate file paths and
indentation. xref-toggle-abbreviation is always an option,

Screens are big nowadays. Standard code is still 80 chars including
indentation.

 Vitalie

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 00:34:08 +0300 wrote:

 > I'd rather not concern myself with this kind of complexity. However,
 > if you're fine with merge logic simply concatenating the results (and
 > it will be hard to do better), this shouldn't be hard to implement on
 > top of the basic one-backend-at-a-time xref system, in a minor mode.

Sure thing. I can do the mode. How about the basic merging? Would it be
possible for you to put some basic stuff together?


  Vitalie

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/05/2015 03:13 AM, Vitalie Spinu wrote:

> Sure thing. I can do the mode. How about the basic merging? Would it be
> possible for you to put some basic stuff together?

What kind of stuff do you have in mind?

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/05/2015 03:02 AM, Vitalie Spinu wrote:
> xref-toggle-abbreviation is always an option,

And it would remove everything but the line content? So I'd toggle it on 
and off in order to know both?

> Screens are big nowadays. Standard code is still 80 chars including
> indentation.

Fully maximized Emacs frame is about 230 columns wide on my laptop. That 
allows to have two 113 column windows side-by-side. I'm pretty sure, 
with file paths and any other columns, the table width with go over 
those 113 columns.

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 03:14:47 +0300 wrote:

 > On 05/05/2015 03:13 AM, Vitalie Spinu wrote:
 >> Sure thing. I can do the mode. How about the basic merging? Would it
 >> be possible for you to put some basic stuff together?

 > What kind of stuff do you have in mind?

I have in mind xref-find-functions list of functions instead of
xref-find-function symbol. But that's tricky because you have
xref-identifier-at-point-function and
xref-identifier-completion-table-function and potentially many more
coming soon. Each of these should also be lists to allow for multiple
backends.

I think a more condensed system would serve us much better here. Instead
of the above 3 config vars use one - `xref-backends` holding a list of
backend names. Like '(elisp etags). Then dynamically expand each of them
as in `elisp-xref-find`, `eslip-xref-identifier-at-point` and
`elisp-xref-completion-table`. If some of the these are not defined use
the default.

Then the minor mode as I have just proposed would be trivial to
implement. I don't see a straightforward implementation based on the
current xref.


  Vitalie





  

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 03:18:17 +0300 wrote:

 > On 05/05/2015 03:02 AM, Vitalie Spinu wrote:
 >> xref-toggle-abbreviation is always an option,

 > And it would remove everything but the line content? So I'd toggle it
 > on and off in order to know both?

That's interesting idea. Yes, I would like that (probably leaving just
the line numbers).

 >> Screens are big nowadays. Standard code is still 80 chars including
 >> indentation.

 > Fully maximized Emacs frame is about 230 columns wide on my
 > laptop. That allows to have two 113 column windows side-by-side. I'm
 > pretty sure, with file paths and any other columns, the table width
 > with go over those 113 columns.

240 on mine. I am rarely bothered with truncated lines with grep or
etags. It's commonly enough for me to see the beginning of the line. And
I am using M-g n and M-g p heavily. When I need full lines I simply
maximize the window. Toggling visibility would be much nicer
though. Especially if it could be done from outside of *xref* buffer
(M-g v?).


  Vitalie

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/05/2015 01:21 AM, Ted Zlatanov wrote:

> If we agree to implement a `next-error-priority', then perhaps
> `next-error-find-buffer' should penalize buried buffers by subtracting
> from their priority.  Thus buffers most recently buried will have the
> lowest priority.

Well yes, if we add a new variable buffers like *compile* set, we could 
detect them easily. And if we not only penalized but also removed 
buffers with priority 0, the list should be short enough.

But would we really have any priority values aside from 0 and 100? I 
think it makes sense for a buffer to only contain errors for itself, or 
only for other buffers.

Re: conflicting uses of next-error-function

[Ted Zlatanov]

On Tue, 5 May 2015 05:28:36 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> On 05/05/2015 01:21 AM, Ted Zlatanov wrote:
>> If we agree to implement a `next-error-priority', then perhaps
>> `next-error-find-buffer' should penalize buried buffers by subtracting
>> from their priority.  Thus buffers most recently buried will have the
>> lowest priority.

DG> Well yes, if we add a new variable buffers like *compile* set, we
DG> could detect them easily. And if we not only penalized but also
DG> removed buffers with priority 0, the list should be short enough.

Right.

DG> But would we really have any priority values aside from 0 and 100? I
DG> think it makes sense for a buffer to only contain errors for itself,
DG> or only for other buffers.

So far, we only know of those two cases, yes.  But we can adjust the
priority further by the buffer's stack position as I suggested above, or
by user-supplied criteria.  We could make it a floating-point value but
I'm not sure that would be a usability gain :)

Ted

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/05/2015 12:47 PM, Ted Zlatanov wrote:

> So far, we only know of those two cases, yes.  But we can adjust the
> priority further by the buffer's stack position as I suggested above, or
> by user-supplied criteria.  We could make it a floating-point value but
> I'm not sure that would be a usability gain :)

My instinct is to have a boolean var, and in case any new cases turn up, 
add a few new possible symbol values. For all we know, non-0, non-100 
situations might not be easily described with a number.

Re: conflicting uses of next-error-function

[Ted Zlatanov]

On Tue, 5 May 2015 17:05:43 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> On 05/05/2015 12:47 PM, Ted Zlatanov wrote:
>> So far, we only know of those two cases, yes.  But we can adjust the
>> priority further by the buffer's stack position as I suggested above, or
>> by user-supplied criteria.  We could make it a floating-point value but
>> I'm not sure that would be a usability gain :)

DG> My instinct is to have a boolean var, and in case any new cases turn
DG> up, add a few new possible symbol values. For all we know, non-0,
DG> non-100 situations might not be easily described with a number.

Sound good to me :)

Ted

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/05/2015 04:36 AM, Vitalie Spinu wrote:
> Each of these should also be lists to allow for multiple
> backends.

List values don't make sense outside of the context of your minor mode, 
so I'd rather you do an external adapter converting a list to one backend.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

Eli,

Would you say there's something left to fix here? Otherwise, it's 
probably time to close it.

bug#19468: 25.0.50; UI inconveniences with M-.

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 22 Feb 2016 01:00:06 +0200
> 
> Eli,
> 
> Would you say there's something left to fix here? Otherwise, it's 
> probably time to close it.

Feel free to close.  I use the feature all the time since the pretest
started, and I think it looks good, thanks.

bug#19468: 25.0.50; UI inconveniences with M-.

[Dmitry Gutov]

On 02/22/2016 07:18 PM, Eli Zaretskii wrote:

> Feel free to close.  I use the feature all the time since the pretest
> started, and I think it looks good, thanks.

That's great to hear. Closing.