Re: UI inconveniences with M-.

Re: UI inconveniences with M-.

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
>> > and if you type the same one, you get nowhere.  The other matches are
>> > only available via completion, see below.
>> Maybe we should write an xref-old-ui-mode.
> I didn't ask for the old UI.  (If I want it, I can still have it,
> since the etags.el commands are still available.)  I'm okay with the
> new UI.  This bug report is about the new UI, and its goal is to make
> the new UI better.

Then I don't understand what you mean by the text I quoted.  E.g.:

   if you try "C-u M-.", you get prompted for the symbol again,
   and if you type the same one, you get nowhere

Well, duh!  What did you expect?  In the new UI, "C-u M-." jumps to
*the* definition (the "C-u" is only used to let you type the name of
thing you're looking for), so of course if you just jumped to it, asking
to jump to it again will get you nowhere else.  I guess it would be OK
to make M-. emit a message like "Hello?  We're already here!" in this
corner case, if you want to make sure something happens.

> old UI was consistent and complete: it always displayed the first/next
> match and provided a command to go to the next/previous one.  By
> contrast, the new UI is inconsistent: with some back-ends it shows the
> list of matches and allows to navigate it, with others it shows only
> the first match and does not give any way to get the next match.

In the "M-. find-tag" example, the UI does let you see all matches.
It's just that there's only one.  If you want, we could make this case
popup an *xref* buffer with a single entry, but I think it makes more
sense to just jump straight to that one entry instead.

> But that's largely immaterial: this bug report is not about the
> back-end, it's about the UI.

I wouldn't be so sure.

>> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>> if org-ctags.el happens to be loaded
> ??? Why should the xref matches depend on what is and isn't loaded?

Because that's how xref/elisp works.

> That would make xref exactly equivalent to "M-x apropos", which means
> this mode of operation would make xref entirely redundant.

To you, maybe.  Personally, I find it's *much* quicker to use M-. than
to use C-h f RET <go-to-the-link> RET, and it has the added benefit that
M-, brings me right back if I need to.

> To say nothing of the fact that this doesn't scale to any language
> except ELisp.

Not at all.  Many (most?) packages which offer a functionality along the
lines of "M-. to jump to the definition" use an implementation technique
which is fundamentally similar/equivalent (obviously, they don't query
a running Emacs, but they query a running REPL).

> We could offer this mode as an optional feature, but it certainly
> shouldn't be the default.

Many users here disagree.

> (One of my worst annoyances is to type a C-h command and be presented
> with "[No match]", because some package is not loaded or some function
> is not available in the Emacs configuration I happen to be using.)

Yes, that's a long standing problem, indeed.  I have a local hack which
tries to add to loaddefs.el a concise description of where definitions
might be found (basically, for each file, list the few prefixes used in
that file), so that "C-h f" can load those packages when needed.
It's "proof of concept" and it has occasionally rendered my Emacs
session unusable (e.g. because it ended up deciding to try and load,
say, w32-fns.el to see if it might contain the definition).  I think
I've now solved most of those problems in the w32-* files, but the
underlying risk is still there.

> That's one way of looking at it.  Another is to say that etags gave
> you more information and thus allowed to find more loose matches,
> which is helpful when your memory is failing you.

M-. jumps to the definition of the "thing under point".  There's no
memory involved.
If you're not sure what you're looking for, then you're expected to use
the completion facilities in C-u M-.

> But in the context of this bug report, that, too, is immaterial: if we
> think the etags back-end gives too much information, by all means
> let's filter it before presenting the matches.

Yes, that's clearly very much needed.

> Bonus points for making the filtering optional, since some people
> might like that, and in some situations even you might need it.

I disagree.  The filtering is needed by the design of the xref API.
If you want to see the "unfiltered" data, then use M-x xref-find-apropos.

> Then let's fix the results we display with the etags back-end to show
> only the relevant ones.

Yes, please.

> Saying that the back-end returns a confusingly large amount of bogus
> matches, and then switching to a UI that assumes much smarter
> back-ends (which don't really exist) makes very little sense to me.

The smarter backends already exist, in so far as there is already code out
there which provides the needed functionality.  It just doesn't use the
xref API yet, because it was written before xref.el.

> The old code attempted to support both use cases, by showing the exact
> match(es) first, followed by less likely ones.  We do similar things
> in other commands.  The advantages of such a method are that (1) you
> don't need to second-guess the user, (2) you provide only one command
> for the user to remember, and (3) if the user only _thinks_ she knows
> the name, but really doesn't, she can find the information without
> having to invoke another command.

>> I think that's an important feature of the new code in this respect.

> But it is incomplete without a means to get to the other possible
> matches in this case.

> To summarize my points in this sub-thread:

>   . the UI should depend much less on the back-end, ideally not at all

You keep repeating this, but that is absolutely meaningless to me (kind
of like "it doesn't work" in bug reports):
- This consistency was trivially obtained with the old etags.el code
  since it had one 1 backend.
- What the fuck should the new UI do if one backend returns 1 match and
  another returns a hundred (as in your "find-tag" example)?

>   . there should be a way to go to the next/previous match if the
>     *xref* buffer is not displayed (or not created in the first
>     place); if this happens because there's only one match, we should
>     say so explicitly

The only case where the *xref* is not displayed is when there's only
1 match returned by the backend.  So "go to the next/previous match" is
again meaningless.

>   . when there are more than one possible match, the UI should behave
>     similarly, i.e. display the *xref* buffer; when there's only one
>     match, it should _never_ display *xref*,

That's exactly what the code does.

>     and should display an indication of the fact that there's only one
>     match

The current code "displays" this indication by jumping to the sole match
instead of jumping to the *xref* buffer.  I think it's clear enough.

>   . if the criteria for filtering of the matches are very different
>     between the possible back-ends, there should be some agreed-upon
>     uniform default that returns roughly the same number of matches
>     with all back-ends, and the rest should be controlled by user
>     options

I don't see what that could concretely look like.


        Stefan

Etags.el (was: UI inconveniences with M-.)

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

>> I really think the etags backend should only return data when the TAGS
>> file is in one of the parents of the current directory.
> That goes back to the "working on several projects" discussion of
> yore.  Specifically, you might be working on two related packages
> (e.g., one calls the other) that live in two different directories.

And as mentioned back then, this would be solved by providing a way for
the user to tell etags.el about the link between those two projects.
For the case where this link only exists during the running session, we
could simply use the existing "visit-tags-table" functionality where you
can add a TAGS file to the one(s) already loaded, so the new
functionality would behave similarly to the old one.

>> And it should also be able to keep several independent TAGS tables
>> opened for different projects in different directory trees.
> It already does that.

No it doesn't.  It lets you use the union of a set of TAGS tables, which
means that the TAGS tables of independent projects pollute each other.


        Stefan

Re: UI inconveniences with M-.

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

> Hmm, maybe we should introduce a new category, for reading symbols.  Aside
> from xref, we could use it in describe-function/variable/etc.

Sounds fine to me.  Tho "symbol" might be a tad too generic.

> Although the fact that it tries prefix matches first, and only falls back to
> substring matches, doesn't look too terrible to me.

Not sure what you're saying here.  Indeed, for buffers,
completion-category-defaults says to first try prefix matching and
fallback on substring only if that fails.  This was done mostly to
reduce friction when the change was introduced.


        Stefan

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:13 AM, Stefan Monnier wrote:

> Sounds fine to me.  Tho "symbol" might be a tad too generic.

...identifier? Better name welcome.

> Not sure what you're saying here.  Indeed, for buffers,
> completion-category-defaults says to first try prefix matching and
> fallback on substring only if that fails.  This was done mostly to
> reduce friction when the change was introduced.

I was saying that about the current identifier completion in `C-u M-.'. 
And was mistaken: it's close, but uses partial-completion (according to 
`completion-styles'). Which is close, but a bit less user-friendly.

Re: UI inconveniences with M-.

[Stefan Monnier]

>> Sounds fine to me.  Tho "symbol" might be a tad too generic.
> ...identifier? Better name welcome.

I originally suggested "identifier" and then changed my mind before
sending the message: the problem, really, is that things like M-x
completion also complete identifiers/symbols yet I'm not convinced they
should use the same category.
Then again, maybe they should.

> I was saying that about the current identifier completion in `C-u M-.'. And
> was mistaken: it's close, but uses partial-completion (according to
> `completion-styles'). Which is close, but a bit less user-friendly.

Of course, a better option might be a new style `pc-substring', which
combines partial completion and substring, so that "f-t" matches not
only things like "find-tag" but also "buffer-face-toggle".

That's been on the "todo" for "ever", but I just got used to typing
"*f-t" or "-f-t" when I need this kind of functionality.


        Stefan

Re: UI inconveniences with M-.

[Helmut Eller]

On Tue, Apr 28 2015, Stefan Monnier wrote:

> [ Moving this to emacs-devel.  ]
>
>>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
>>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
>>> > and if you type the same one, you get nowhere.  The other matches are
>>> > only available via completion, see below.

I my proposal the next-error/prev-error commands could be used to
navigate through xrefs even after the *xref* buffer disappeared.  This
can was done--quite elegantly I think--by setting next-error-last-buffer
to the *xref* buffer and set the buffer local value of
next-error-function in the *xref* buffer appropriately.

Dmitry took that out because he thinks that it interferes too much with
compilation-mode.  Maybe he's right but let me note that M-x rgrep also
uses next-error/prev-error and similarly interferes with
compilation-mode.

Helmut

conflicting uses of next-error-function (was: UI inconveniences with M-.)

[Stefan Monnier]

> Dmitry took that out because he thinks that it interferes too much with
> compilation-mode.  Maybe he's right but let me note that M-x rgrep also
> uses next-error/prev-error and similarly interferes with
> compilation-mode.

Indeed, I've occasionally been annoyed by conflicting uses of
next-error-last-buffer/next-error-function.  And in both directions:
sometimes I don't want `grep' to hijack my ongoing traversal of
compiler warnings, and sometimes I regret that some other package
doesn't let me use C-x `

Maybe we should consider refining this so that all package which can
sometimes benefit from C-x ` would feel free to provide support for
next-error, without the fear of imposing themselves.  For that, the
next-error framework needs to make it possible for the user to control
which package gets to really use next-error and when.

Not sure what that should look like, tho.  Maybe some config var could
specify which packages's next-error-support is "enabled" by default,
then some way for the user to override that default on a case by case
basis: e.g. a command to run inside the *xref* buffer would
"force-enable" the next-error support for this one time, and maybe
a new `next-error-pop-context' command would go back to the previous
"next-error-last-buffer/function"?


        Stefan

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Helmut Eller on Wed, 29 Apr 2015 08:10:45 +0200 wrote:

 > On Tue, Apr 28 2015, Stefan Monnier wrote:
 >> [ Moving this to emacs-devel.  ]
 >> 
 >>>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
 >>>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
 >>>> > and if you type the same one, you get nowhere.  The other matches are
 >>>> > only available via completion, see below.

 > I my proposal the next-error/prev-error commands could be used to
 > navigate through xrefs even after the *xref* buffer disappeared.  This
 > can was done--quite elegantly I think--by setting next-error-last-buffer
 > to the *xref* buffer and set the buffer local value of
 > next-error-function in the *xref* buffer appropriately.

 > Dmitry took that out because he thinks that it interferes too much with
 > compilation-mode.  

How does it interfere with the compilation-mode more concretely?

By definition next-error(M-g n)/previous-error(M-g p) operate on the
most recent buffer (in buffer-list) that contains `next-error-function`
and is complete unrelated to compilation mode. If you just called
`occur` you navigate in *occur* buffer, grep - in *grep*, compile - in
*compile* etc.

IMW *xref* is conceptually closer to *compile* than to *occur* or even
*grep* buffers.

Dmitry, would you mind putting this back?

Thanks,

  Vitalie

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Tue, 28 Apr 2015 22:41:27 -0400
> 
> >> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
> >> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
> >> > and if you type the same one, you get nowhere.  The other matches are
> >> > only available via completion, see below.
> >> Maybe we should write an xref-old-ui-mode.
> > I didn't ask for the old UI.  (If I want it, I can still have it,
> > since the etags.el commands are still available.)  I'm okay with the
> > new UI.  This bug report is about the new UI, and its goal is to make
> > the new UI better.
> 
> Then I don't understand what you mean by the text I quoted.  E.g.:
> 
>    if you try "C-u M-.", you get prompted for the symbol again,
>    and if you type the same one, you get nowhere

If this part of my response gets in the way, just disregard it.
(FWIW, it was an attempt to demonstrate that there's no functionality
in the new UI that is equivalent to "C-u M-." in the old UI.  But it's
not an important point, so feel free to forget about it.)

> > old UI was consistent and complete: it always displayed the first/next
> > match and provided a command to go to the next/previous one.  By
> > contrast, the new UI is inconsistent: with some back-ends it shows the
> > list of matches and allows to navigate it, with others it shows only
> > the first match and does not give any way to get the next match.
> 
> In the "M-. find-tag" example, the UI does let you see all matches.
> It's just that there's only one.

We already agreed that there were at least 2.

More importantly, the issue that I was raising in that example is that
the results change too radically when the back-end is changed.  IMO,
it makes no sense to display more than 140 candidates with one
back-end, and only one with another, in the default configuration.

> If you want, we could make this case popup an *xref* buffer with a
> single entry

Once again, I never said that a single match should be popped in a
buffer.  I said that I don't expect to see a single match in this
case, because that's a lie!

> > But that's largely immaterial: this bug report is not about the
> > back-end, it's about the UI.
> 
> I wouldn't be so sure.

I am.  We keep discussing the back-ends because you and Dmitry
consistently shift the discussion in that direction, so I feel
compelled to respond.  But my perspective in filing the bug report was
the perspective of a user looking at the results through the UI, and
trying to use that UI to manipulate the results.

If some changes are needed in the back-ends to make the UI be better,
then so be it.  But it is not a requirement, and I don't feel I know
enough at this stage to even talk intelligently which changes are
needed and in what parts of this puzzle.

> >> Arguably, the find-tag advice in org-ctags.el could be offered as well,
> >> if org-ctags.el happens to be loaded
> > ??? Why should the xref matches depend on what is and isn't loaded?
> 
> Because that's how xref/elisp works.

If so, then it's much less useful than I thought.  I don't see how
such a back-end can even be a candidate for becoming the default.

> > That would make xref exactly equivalent to "M-x apropos", which means
> > this mode of operation would make xref entirely redundant.
> 
> To you, maybe.  Personally, I find it's *much* quicker to use M-. than
> to use C-h f RET <go-to-the-link> RET

So adding the default guessing depending on what's at point is all
that you need?

> and it has the added benefit that M-, brings me right back if I need
> to.

Apropos pops up a new window, and 'q' quits it, so that problem has a
solution there as well.

> > To say nothing of the fact that this doesn't scale to any language
> > except ELisp.
> 
> Not at all.  Many (most?) packages which offer a functionality along the
> lines of "M-. to jump to the definition" use an implementation technique
> which is fundamentally similar/equivalent (obviously, they don't query
> a running Emacs, but they query a running REPL).

Think about compiled languages.

> > We could offer this mode as an optional feature, but it certainly
> > shouldn't be the default.
> 
> Many users here disagree.

If those many use mainly ELisp or REPL-type language interpreters, I
can understand why.  But Emacs is not limited to those.

> > (One of my worst annoyances is to type a C-h command and be presented
> > with "[No match]", because some package is not loaded or some function
> > is not available in the Emacs configuration I happen to be using.)
> 
> Yes, that's a long standing problem, indeed.  I have a local hack which
> tries to add to loaddefs.el a concise description of where definitions
> might be found (basically, for each file, list the few prefixes used in
> that file), so that "C-h f" can load those packages when needed.
> It's "proof of concept" and it has occasionally rendered my Emacs
> session unusable (e.g. because it ended up deciding to try and load,
> say, w32-fns.el to see if it might contain the definition).  I think
> I've now solved most of those problems in the w32-* files, but the
> underlying risk is still there.

Until that problem is solved in core, I don't see how a back-end that
requires to load a package can be the default.

> > That's one way of looking at it.  Another is to say that etags gave
> > you more information and thus allowed to find more loose matches,
> > which is helpful when your memory is failing you.
> 
> M-. jumps to the definition of the "thing under point".  There's no
> memory involved.

I was talking about "C-u M-."

> If you're not sure what you're looking for, then you're expected to use
> the completion facilities in C-u M-.

Completion shows only part of the matches, we've been through that
already.  It doesn't by default show substring matches.

> > Bonus points for making the filtering optional, since some people
> > might like that, and in some situations even you might need it.
> 
> I disagree.

Then you won't get bonus points from me.

> The filtering is needed by the design of the xref API.
> If you want to see the "unfiltered" data, then use M-x xref-find-apropos.

I didn't say "unfiltered".  I asked for user control of the amount of
filtering or of looseness of the matches.  In etags.el terms, think
about giving the user control on what goes into find-tag-tag-order.

> > Saying that the back-end returns a confusingly large amount of bogus
> > matches, and then switching to a UI that assumes much smarter
> > back-ends (which don't really exist) makes very little sense to me.
> 
> The smarter backends already exist

Where are they?

> in so far as there is already code out there which provides the
> needed functionality.  It just doesn't use the xref API yet, because
> it was written before xref.el.

Ah, so they don't really exist.

Once again, I question the decision to switch to this UI, when we
clearly didn't yet get our act together and are not really prepared to
use it.  The _potential_ to use it exists, but it is currently not
usable.  We should have waited until it was, IMO.

> >   . the UI should depend much less on the back-end, ideally not at all
> 
> You keep repeating this, but that is absolutely meaningless to me (kind
> of like "it doesn't work" in bug reports):
> - This consistency was trivially obtained with the old etags.el code
>   since it had one 1 backend.
> - What the fuck should the new UI do if one backend returns 1 match and
>   another returns a hundred (as in your "find-tag" example)?

It depends on which one is closer to the correct result, obviously.
If 100 is closer, fix or rewrite (or throw away) the back-end that
returns only 1; if 1 is closer, filter the 100 results to return less
than 10.

But having 2 orders of magnitude difference between the results
_by_default_ is absolutely insane!

> >   . there should be a way to go to the next/previous match if the
> >     *xref* buffer is not displayed (or not created in the first
> >     place); if this happens because there's only one match, we should
> >     say so explicitly
> 
> The only case where the *xref* is not displayed is when there's only
> 1 match returned by the backend.  So "go to the next/previous match" is
> again meaningless.

It's not!!  There are partial matches shown by completion -- they are
the next candidates.  If we are not sure the user will want them, we
can ask for confirmation, like we do, for example, in Info's search
that fails to find a match and needs to cross the node boundary.

> >     and should display an indication of the fact that there's only one
> >     match
> 
> The current code "displays" this indication by jumping to the sole match
> instead of jumping to the *xref* buffer.  I think it's clear enough.

We do show similar indications in other cases.  Again, Info's 'i'
comes to mind.

More generally, a UI that does nothing and expects the user to
understand what it means is IMO a bad UI.  At the very least, it
leaves the user wondering whether she hit a bug.

> >   . if the criteria for filtering of the matches are very different
> >     between the possible back-ends, there should be some agreed-upon
> >     uniform default that returns roughly the same number of matches
> >     with all back-ends, and the rest should be controlled by user
> >     options
> 
> I don't see what that could concretely look like.

User options to control how loose the matches are, and the default
level of looseness that yields similar results.

Re: Etags.el (was: UI inconveniences with M-.)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Tue, 28 Apr 2015 23:12:53 -0400
> 
> >> I really think the etags backend should only return data when the TAGS
> >> file is in one of the parents of the current directory.
> > That goes back to the "working on several projects" discussion of
> > yore.  Specifically, you might be working on two related packages
> > (e.g., one calls the other) that live in two different directories.
> 
> And as mentioned back then, this would be solved by providing a way for
> the user to tell etags.el about the link between those two projects.
> For the case where this link only exists during the running session, we
> could simply use the existing "visit-tags-table" functionality where you
> can add a TAGS file to the one(s) already loaded, so the new
> functionality would behave similarly to the old one.

And as mentioned back then, until something like this is coded, this
discussion is just theoretical.

> >> And it should also be able to keep several independent TAGS tables
> >> opened for different projects in different directory trees.
> > It already does that.
> >
> No it doesn't.  It lets you use the union of a set of TAGS tables, which
> means that the TAGS tables of independent projects pollute each other.

It's an approximation, yes.  Feel free to implement something better;
until then, what we have is quite good IME (I use it every day).

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:44 PM, Eli Zaretskii wrote:

> It depends on which one is closer to the correct result, obviously.
> If 100 is closer, fix or rewrite (or throw away) the back-end that
> returns only 1; if 1 is closer, filter the 100 results to return less
> than 10.
>
> But having 2 orders of magnitude difference between the results
> _by_default_ is absolutely insane!

How would a UI deal with that?

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 18:57:59 +0300
> 
> On 04/29/2015 06:44 PM, Eli Zaretskii wrote:
> 
> > It depends on which one is closer to the correct result, obviously.
> > If 100 is closer, fix or rewrite (or throw away) the back-end that
> > returns only 1; if 1 is closer, filter the 100 results to return less
> > than 10.
> >
> > But having 2 orders of magnitude difference between the results
> > _by_default_ is absolutely insane!
> 
> How would a UI deal with that?

I don't know yet.  For now, I'm having hard time getting you-all to
agree that this needs to be fixed, and fixed soon.  Are we there yet?

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 07:17 PM, Eli Zaretskii wrote:

>> How would a UI deal with that?
>
> I don't know yet.  For now, I'm having hard time getting you-all to
> agree that this needs to be fixed, and fixed soon.  Are we there yet?

Not really. You seem to insist on fixing this in UI. This isn't far, 
conceptually, from teaching pig how to fly, from where I'm standing.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 19:25:35 +0300
> 
> On 04/29/2015 07:17 PM, Eli Zaretskii wrote:
> 
> >> How would a UI deal with that?
> >
> > I don't know yet.  For now, I'm having hard time getting you-all to
> > agree that this needs to be fixed, and fixed soon.  Are we there yet?
> 
> Not really. You seem to insist on fixing this in UI.

I never said that.  What I did say is that the problems exposed by the
UI need to be fixed, no matter where.

> This isn't far, conceptually, from teaching pig how to fly, from
> where I'm standing.

Thank you for your kind words.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Wed, Apr 29 2015, Stefan Monnier wrote:

> Not sure what that should look like, tho.  Maybe some config var could
> specify which packages's next-error-support is "enabled" by default,
> then some way for the user to override that default on a case by case
> basis: e.g. a command to run inside the *xref* buffer would
> "force-enable" the next-error support for this one time, and maybe
> a new `next-error-pop-context' command would go back to the previous
> "next-error-last-buffer/function"?

Maybe we could mark a buffer somehow that it's a reasonable candidate for
next-error-last-buffer and have a command ala switch-buffer to select
the current next-error-buffer from the list of candidates.

Helmut

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 07:53 PM, Eli Zaretskii wrote:

> I never said that.  What I did say is that the problems exposed by the
> UI need to be fixed, no matter where.

I full-heartedly agree that we should fix problems.

Let me quote you here:

"But that's largely immaterial: this bug report is not about the
back-end, it's about the UI."

"We keep discussing the back-ends because you and Dmitry
consistently shift the discussion in that direction, so I feel
compelled to respond.  But my perspective in filing the bug report was
the perspective of a user looking at the results through the UI"

So you seem to use the term "UI" to refer to the whole user experience 
(as opposed to the UI part of xref, which does not include the 
backends). This might be easier for you, but it leads to confusion, 
because the rest of us are trying to discuss it at a more detailed level.

> Thank you for your kind words.

No offence intended.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 09:10 AM, Helmut Eller wrote:

> Dmitry took that out because he thinks that it interferes too much with
> compilation-mode.  Maybe he's right but let me note that M-x rgrep also
> uses next-error/prev-error and similarly interferes with
> compilation-mode.

Sorry, I guess. I brought up the problem, and didn't see it sufficiently 
addressed or even discussed, so I dropped that part of the code.

Since some discussion is happening how (and users have expressed 
interest in it either way), I'll reinstate the code.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 20:06:07 +0300
> 
> On 04/29/2015 07:53 PM, Eli Zaretskii wrote:
> 
> > I never said that.  What I did say is that the problems exposed by the
> > UI need to be fixed, no matter where.
> 
> I full-heartedly agree that we should fix problems.
> 
> Let me quote you here:
> 
> "But that's largely immaterial: this bug report is not about the
> back-end, it's about the UI."
> 
> "We keep discussing the back-ends because you and Dmitry
> consistently shift the discussion in that direction, so I feel
> compelled to respond.  But my perspective in filing the bug report was
> the perspective of a user looking at the results through the UI"

Where does this say that the problems need to be fixed in the UI?

> So you seem to use the term "UI" to refer to the whole user experience 
> (as opposed to the UI part of xref, which does not include the 
> backends). This might be easier for you, but it leads to confusion, 
> because the rest of us are trying to discuss it at a more detailed level.

I'm sorry that I confused you, and glad that we have that sorted out
now.

> > Thank you for your kind words.
> 
> No offence intended.

You could fool me.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 08:15 PM, Eli Zaretskii wrote:

> Where does this say that the problems need to be fixed in the UI?

In "this bug report is ... about the UI".

> I'm sorry that I confused you, and glad that we have that sorted out
> now.

Ok, thanks.

> You could fool me.

Was that a wrong idiom? I only meant to state that "fixing the UI" is 
more or less impossible.

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/29/2015 07:58 PM, Helmut Eller wrote:

> Maybe we could mark a buffer somehow that it's a reasonable candidate for
> next-error-last-buffer and have a command ala switch-buffer to select
> the current next-error-buffer from the list of candidates.

Yes, that sounds like the easiest approach. Then using 
next-error-function in xref at least won't make things worse.

The "mark a buffer somehow that it's a reasonable candidate" is an 
important step, because even plain buffers can have local 
next-error-function values, set by Flycheck or Flymake, for instance.

How will we do that?

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 29 Apr 2015 20:26:18 +0300
> 
> > You could fool me.
> 
> Was that a wrong idiom?

It was an offensive metaphor.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 08:59 PM, Eli Zaretskii wrote:

> It was an offensive metaphor.

Well, please excuse any offence. It was intended to be humorous.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Wed, Apr 29 2015, Dmitry Gutov wrote:

> The "mark a buffer somehow that it's a reasonable candidate" is an
> important step, because even plain buffers can have local
> next-error-function values, set by Flycheck or Flymake, for instance.
>
> How will we do that?

I would just use a buffer local variable, like next-error-buffer-p, or
perhaps somebody can come up with a more catchy name.

It might be nifty (but more complicated) to organize those buffers as a
stack or ring so that a context can be pushed/popped; if so then it
might be useful to use an integer instead of a boolean as value for this
buffer local variable.

Helmut

Re: UI inconveniences with M-.

[Stefan Monnier]

>> In the "M-. find-tag" example, the UI does let you see all matches.
>> It's just that there's only one.
> We already agreed that there were at least 2.

No, we agreed that in some ideal world there could be two.  But the
backend as it is written now, only returns one for that case.

> More importantly, the issue that I was raising in that example is that
> the results change too radically when the back-end is changed.  IMO,
> it makes no sense to display more than 140 candidates with one
> back-end, and only one with another, in the default configuration.

I agree the 140 "matches" are absurd and need to be fixed.
The single match of the other backend is what I want, OTOH.

> Once again, I never said that a single match should be popped in a
> buffer.  I said that I don't expect to see a single match in this
> case, because that's a lie!

No, it's not a lie and I would even argue it's neither a bug nor
a misfeature.  You expect etags-style information from the xref/elisp
backend, but that's simply not the info it tries to give you.

> compelled to respond.  But my perspective in filing the bug report was
> the perspective of a user looking at the results through the UI, and
> trying to use that UI to manipulate the results.

See, you use the term "UI" in a completely different way.  When I say
"UI" in this discussion, I'm talking about the code which takes the info
from the backend and makes it available to the user.

So the fact that one backend returns 1 item and the other returns 140 of
them is completely outside of the control of the UI.
Maybe the resulting end-user experience sucks, but that's not the fault
of the UI but the fault of the backends.

The reason why I use the term UI in this way is because the API is
designed so that you can have several UIs using the same API (just like
you can have several backends).

>> >> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>> >> if org-ctags.el happens to be loaded
>> > ??? Why should the xref matches depend on what is and isn't loaded?
>> Because that's how xref/elisp works.
> If so, then it's much less useful than I thought.

And to me, it makes it more useful.

> I don't see how such a back-end can even be a candidate for becoming
> the default.

And I don't see how we've live with the etags.el UI for so many years.
I've tried several times to make real use of it, but always found it
completely unpalatable.  What with having to build those damn TAGS
files, remember to refresh them, remember where they are, constantly
tell Emacs again where they are, deal with its inability to find the
right spot and having to repeat the "C-u M-." finger gymnastics
umpteen times.

xref/elisp may not let you jump to those functions/vars that aren't yet
loaded, but beside this little detail, it *just works* without any
extra setup, nothing.

> Apropos pops up a new window, and 'q' quits it, so that problem has a
> solution there as well.

But the apropos is just an intermediate, what I want to see is the
corresponding source, and by the time I'm there, I can't so easily use
`q' to go back.

>> Not at all.  Many (most?) packages which offer a functionality along the
>> lines of "M-. to jump to the definition" use an implementation technique
>> which is fundamentally similar/equivalent (obviously, they don't query
>> a running Emacs, but they query a running REPL).
> Think about compiled languages.

AFAIK, JDEE used the same approach for Java.

[ There's no such thing as "a compiled language".  I guess you mean
  "think of the case where your language's implementation uses good old
  static compilation scheme".  ]

I don't claim that all backends will have to work this way.  Just that
it's a common implementation technique.

> If those many use mainly ELisp or REPL-type language interpreters, I
> can understand why.  But Emacs is not limited to those.

And neither is the xref API.

> Until that problem is solved in core, I don't see how a back-end that
> requires to load a package can be the default.

Agreed.

>> > That's one way of looking at it.  Another is to say that etags gave
>> > you more information and thus allowed to find more loose matches,
>> > which is helpful when your memory is failing you.
>> M-. jumps to the definition of the "thing under point".  There's no
>> memory involved.
> I was talking about "C-u M-."

C-u M-. also lets you do loose matching, via completion, if your memory
is failing you.

>> If you're not sure what you're looking for, then you're expected to use
>> the completion facilities in C-u M-.
> Completion shows only part of the matches, we've been through that
> already.  It doesn't by default show substring matches.

For one, you can make substring the default by tweaking the completion
styles configuration, if you so prefer.  The default completion is
partial-completion, which is actually "more powerful" than substring
matching: e.g. "*find-tag" would not only match "find-tag-tag" and
"my-own-find-tag" but also "my-finder-tag".

But there's nothing magical about substring matching anyway.  etags.el
used it because it was easy to implement, not because it is the best
thing there is.  Of course, it's good, but partial-completion is also
good, and many other options are also good.

> I didn't say "unfiltered".  I asked for user control of the amount of
> filtering or of looseness of the matches.  In etags.el terms, think
> about giving the user control on what goes into find-tag-tag-order.

That's OK, then.  It seems that etags is simply unable to give
sufficiently reliable data to filter it effectively, so we're stuck with
heuristics and user-knobs to choose where they're rather get bitten.

> Once again, I question the decision to switch to this UI, when we

I know, but that's just a waste of time.  The backends won't come before
this system is standard and common place (and they'll take their time,
because in the mean time they can keep using their ad-hoc UI which also
works on older Emacsen).

> But having 2 orders of magnitude difference between the results
> _by_default_ is absolutely insane!

Agreed, because the etags backend sucks right now.

> It's not!!  There are partial matches shown by completion -- they are
> the next candidates.

Again, you choose to look at it this way because you're used to the way
etags.el worked.  But to me, "find-tag" and "find-tag-tag" are not two
different matches to my request.  They're just two completely unrelated
things: either I'm looking for one or I'm looking for the other.

The design of xref is based on the hope that the backend can return
a very small list of candidates, ideally of length 1.  That's not always
the case, but for most potential backends (other than those based on
heuristic techniques like regexp-matching) it's pretty close.

>> >   . if the criteria for filtering of the matches are very different
>> >     between the possible back-ends, there should be some agreed-upon
>> >     uniform default that returns roughly the same number of matches
>> >     with all back-ends, and the rest should be controlled by user
>> >     options
>> I don't see what that could concretely look like.
> User options to control how loose the matches are, and the default
> level of looseness that yields similar results.

Feel free to implement another UI on top of this API which provides this
functionality.  I sure will stay away from it, because I have no use for
something which wastes my time showing me unrelated functions whose name
just happen to contain the name of my function as a substring.

On the contrary, I want a tool that's precise, and gets me directly to
the only corresponding definition (in the 99% of the cases where
there's indeed only one) with a single key-press and no questions asked.


        Stefan

Re: Etags.el

[Stefan Monnier]

> And as mentioned back then, until something like this is coded, this
> discussion is just theoretical.

Agreed.  I don't use etags.el so I'm not highly motivated, and when I've
had to look at its code, I've found it has a lot of historical baggage,
which makes it more painful than necessary to do most things (e.g. the
redundancy between tags-table-list and tags-file-name).


        Stefan

Re: conflicting uses of next-error-function

[Vitalie Spinu]

 >>> Stefan Monnier on Wed, 29 Apr 2015 09:23:13 -0400 wrote:

 >> Dmitry took that out because he thinks that it interferes too much with
 >> compilation-mode.  Maybe he's right but let me note that M-x rgrep also
 >> uses next-error/prev-error and similarly interferes with
 >> compilation-mode.

 > Indeed, I've occasionally been annoyed by conflicting uses of
 > next-error-last-buffer/next-error-function.  And in both directions:

Shouldn't then the first invocation of next-error push into
xref--marker-ring (for M-.) in case it took you to a wrong place?

 > For that, the next-error framework needs to make it possible for the
 > user to control which package gets to really use next-error and when.
 
How about using the free M-0 prefix in next-error to prompt for the
compilation buffer on which to operate?


   Vitalie

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/30/2015 02:05 AM, Vitalie Spinu wrote:

> Shouldn't then the first invocation of next-error push into
> xref--marker-ring (for M-.) in case it took you to a wrong place?

That would be nice. Something like (unless (eq last-command 'next-error) 
(xref-push-marker-stack)).

> How about using the free M-0 prefix in next-error to prompt for the
> compilation buffer on which to operate?

Unfortunately, it's already taken: both `next-error' and 
`previous-error' take numeric arguments, and `M-0 C-`' means "move by 
zero errors".

Which looks like a waste to me: it pretty rare to move by many errors at 
a time (and know that number precisely).

`C-x 4 C-`'?

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/29/2015 09:15 PM, Helmut Eller wrote:

> It might be nifty (but more complicated) to organize those buffers as a
> stack or ring so that a context can be pushed/popped; if so then it
> might be useful to use an integer instead of a boolean as value for this
> buffer local variable.

Yup, a ring might be a replacement for both next-error-last-buffer and 
next-error-buffer-p. But when will buffers be popped from it?

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/29/2015 04:23 PM, Stefan Monnier wrote:

> and maybe
> a new `next-error-pop-context' command would go back to the previous
> "next-error-last-buffer/function"?

What kind of binding would you see for it?

Re: conflicting uses of next-error-function

[Vitalie Spinu]

 >>> Dmitry Gutov on Thu, 30 Apr 2015 02:11:52 +0300 wrote:

 > On 04/30/2015 02:05 AM, Vitalie Spinu wrote:
 >> Shouldn't then the first invocation of next-error push into
 >> xref--marker-ring (for M-.) in case it took you to a wrong place?

 > That would be nice. Something like (unless (eq last-command 'next-error)
 > (xref-push-marker-stack)).

Yes. But don't offend the `previous-error ;)

 >> How about using the free M-0 prefix in next-error to prompt for the
 >> compilation buffer on which to operate?

 > Unfortunately, it's already taken: both `next-error' and `previous-error' take
 > numeric arguments, and `M-0 C-`' means "move by zero errors".

Hm... I thought that's useless, but it actually makes sense.

 > Which looks like a waste to me: it pretty rare to move by many errors at a time
 > (and know that number precisely).

If after me, numeric prefixes for all navigation commands are pretty a
wasteful. I never use those.

> `C-x 4 C-`'?

It's actually longer "C-x 4 C-x `"'

M-g map is pretty empty. Maybe "M-g e" and M-g M-e? Easy to type on
qwerty.

  Vitalie

(BTW, would be nice to have "M-g c" bound to `goto-char` to keep company
to "M-g g")

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/29/2015 06:26 PM, Vitalie Spinu wrote:

> Dmitry, would you mind putting this back?

And done.

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Thu, 30 Apr 2015 03:44:20 +0300 wrote:

 > On 04/29/2015 06:26 PM, Vitalie Spinu wrote:
 >> Dmitry, would you mind putting this back?

 > And done.

Thanks!

Re: conflicting uses of next-error-function

[Helmut Eller]

On Thu, Apr 30 2015, Dmitry Gutov wrote:

> Yup, a ring might be a replacement for both next-error-last-buffer and
> next-error-buffer-p. But when will buffers be popped from it?

Maybe never, or more precisely: dead buffers in the ring are ignored and
removed automatically.

I don't know what the issue with flymake is but it would certainly be
nice if this would work without having to update existing clients.

Helmut

Re: conflicting uses of next-error-function

[Stefan Monnier]

>> and maybe a new `next-error-pop-context' command would go back to the
>> previous "next-error-last-buffer/function"?
> What kind of binding would you see for it?

I was thinking of one of those brain implants.


        Stefan

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/30/2015 09:19 AM, Helmut Eller wrote:

> I don't know what the issue with flymake is but it would certainly be
> nice if this would work without having to update existing clients.

Flycheck (but not Flymake, actually) sets next-error-function is each 
buffer its minor mode is enabled in.

So, we can't simply collect all buffers where next-error-function is 
set, as candidates for `change-next-error-last-buffer'.

Is there anything else I'm missing that we could do without updating 
existing clients?

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Wed, 29 Apr 2015 17:54:04 -0400
> 
> >> In the "M-. find-tag" example, the UI does let you see all matches.
> >> It's just that there's only one.
> > We already agreed that there were at least 2.
> 
> No, we agreed that in some ideal world there could be two.  But the
> backend as it is written now, only returns one for that case.

If the back-end conceals potentially useful information, it should be
fixed not to do so.

> > More importantly, the issue that I was raising in that example is that
> > the results change too radically when the back-end is changed.  IMO,
> > it makes no sense to display more than 140 candidates with one
> > back-end, and only one with another, in the default configuration.
> 
> I agree the 140 "matches" are absurd and need to be fixed.
> The single match of the other backend is what I want, OTOH.

Only when we are 100% sure there is really only one match relevant to
the user.  When we aren't 100% sure, it's better to show a few
additional candidates and rely on the user to disregard them if she
doesn't need them.

IOW, a bit of additional, perhaps redundant information is much less
of a problem than missing information.

That you personally find that information always redundant does not
mean it's true for everyone else.  There's more than one use case for
these queries, see below.

> > Once again, I never said that a single match should be popped in a
> > buffer.  I said that I don't expect to see a single match in this
> > case, because that's a lie!
> 
> No, it's not a lie and I would even argue it's neither a bug nor
> a misfeature.  You expect etags-style information from the xref/elisp
> backend, but that's simply not the info it tries to give you.

Once again, if the back-end conceals potentially useful information,
the back-end needs to be fixed.

While etags shows a very long list of potential matches, it produces
them in the order of importance; I don't remember ever having to
examine more than the first few hits (and thus wasn't even aware of of
the sheer length of the full list).  So with the old UI the
ridiculously long list of candidates was never in my face.  The new
UI, by contrast, makes this acutely evident by showing me the full
list, and on top of that it destroys the order produced by the
back-end, which would allow me to look only at the few first hits.

IOW, the etags back-end by default produces a long list because it
relies on the old front-end to avoid annoying the user by the length
of that list.  The new UI, by contrast, is clearly designed on the
assumption that the list of candidates is very short, so it must
either limit the number of match strategies used by etags, or filter
the results according to some useful criteria.

> > compelled to respond.  But my perspective in filing the bug report was
> > the perspective of a user looking at the results through the UI, and
> > trying to use that UI to manipulate the results.
> 
> See, you use the term "UI" in a completely different way.  When I say
> "UI" in this discussion, I'm talking about the code which takes the info
> from the backend and makes it available to the user.

That's your misunderstanding.  I described my user experience from
using this new feature; I never said where the fixes should be made,
primarily because I didn't understand the code well enough (and still
don't) to say anything intelligent about it.

If I wanted to talk about the code, I'd say something like
"this-and-that function does wrong things because of so-and-so".

> So the fact that one backend returns 1 item and the other returns 140 of
> them is completely outside of the control of the UI.

Says you.  Through my naive-user eyes, filtering 140 hits to provide
just a few is perfectly within the capabilities of the UI, at least in
principle.

> Maybe the resulting end-user experience sucks, but that's not the fault
> of the UI but the fault of the backends.

It sounds more and more like it could be the fault of the design, and
specifically of how functionality is divided between the back-end and
the UI.  Let me elaborate.

From the rest of the discussion, and the related threads spawned by
it, I seem to get the feeling that the design was explicitly meant to
leave the UI completely oblivious to the subject matter, and instead
infinitely trust the back-end to produce "the right" results.  If so,
I think this is a design mistake, and this discussion (and my
frustrating user experience) is the result.  IME, an interactive
user-level feature that supports multiple back-end engines cannot
avoid some degree of subject-matter knowledge and some kind of
manipulation of the results that come from the back-end, or else it
won't be able to provide consistent user experience across the
supported back-ends.  Having such a "dumb" UI in this case will cause
complications, some of which we already see:

  . the need to "fix" the back-ends, and continue "fixing" them in the
    future
  . the need to merge back-ends and invent some suitable
    infrastructure for that
  . etc.

To me, this means that separating the back-ends from the UI while
leaving the UI "dumb" is basically unworkable, because such a
separation does not really separate anything: there will still be a
very high degree of coherency and cohesion between the two parts.  Or
maybe there will only be one usable back-end, and the rest will
bit-rot.

IOW, the separation of functionality is in the wrong place.  To be
useful, some of the "smarts" need to be on the UI side, where user
control can be best implemented, and where user intent is known much
better.  Then we could stop caring about back-ends providing too much
info, because the filtering will be in the UI and, most importantly,
by uniform criteria.  And even if the filtering is in some of the
back-ends, it will be according to UI's guidance, whereby the UI will
convert the user-level control options into the language
understandable by the back-end.  That is the only practical way of
providing a consistent user experience with multiple back-ends.

> >> >> Arguably, the find-tag advice in org-ctags.el could be offered as well,
> >> >> if org-ctags.el happens to be loaded
> >> > ??? Why should the xref matches depend on what is and isn't loaded?
> >> Because that's how xref/elisp works.
> > If so, then it's much less useful than I thought.
> 
> And to me, it makes it more useful.
> 
> > I don't see how such a back-end can even be a candidate for becoming
> > the default.
> 
> And I don't see how we've live with the etags.el UI for so many years.

How frequently did you need to study a large body of code written by
someone else, like a large project where you need to fix a bug?

I do that a lot, both on my daytime job, and in Free Software related
moonlighting.  If you seldom, or never, have to quickly find your way
in such vast unchartered territories, and mainly work with code that
either you yourself wrote or with which you are intimately familiar,
then I can understand why you don't like more than one match
candidate.

Likewise, people who mainly work with small projects, where the code
line count is in hundreds or a small number of thousands, might never
need the less-than-exact matches provided by etags: Grep is good
enough and fast enough for that.  But you cannot do without etags in a
larger project.  (You also cannot do without being able to find all
references to an identifier, which is why I'm using ID Utils for the
past 15 years, and was disappointed to learn that xref-find-references
has no back-ends that implemented this kind of search.)

I very much hope Emacs will continue to be able to support the kind of
activities I described above, which AFAIK are very important part of a
software developer's job throughout the industry.

> I've tried several times to make real use of it, but always found it
> completely unpalatable.  What with having to build those damn TAGS
> files, remember to refresh them, remember where they are, constantly
> tell Emacs again where they are, deal with its inability to find the
> right spot and having to repeat the "C-u M-." finger gymnastics
> umpteen times.

Those are exaggerations.  Building TAGS is almost instant, even in
Emacs, you need only refresh them very seldom, and Emacs offers the
place from which to load them so you don't need to remember.

But this, again, is immaterial for this discussion.  I hope you will
agree that, whatever issues we have with etags, replacing it with
something that lacks important functionality is not a good idea.

> xref/elisp may not let you jump to those functions/vars that aren't yet
> loaded, but beside this little detail, it *just works* without any
> extra setup, nothing.

That "little detail" all but invalidates most of my use cases.  We
seem to have very different development experiences, probably due to
what each one of us does most of the time, but I hope you can see that
there are valid use cases outside of your personal experience, and
Emacs should support them if it wants to be a good IDE.

And I'm not even sure your ideas of how to solve that "little detail"
are workable, because of the potentially adverse consequences of
loading code you don't actually need (or want) to execute.  What if
the code is buggy, or dangerous, or simply does things you don't want
to be done in your Emacs session?

And how do you scale this up to packages that are outside of
load-path, for whatever reasons?  Etc. etc.

> >> Not at all.  Many (most?) packages which offer a functionality along the
> >> lines of "M-. to jump to the definition" use an implementation technique
> >> which is fundamentally similar/equivalent (obviously, they don't query
> >> a running Emacs, but they query a running REPL).
> > Think about compiled languages.
> 
> AFAIK, JDEE used the same approach for Java.

I don't know what that means, and don't know enough about JDEE to talk
about it.  In any case, Java is not a "classic" compiled language, as
a Jave development environment is generally capable of running the
code in an interpreter.

> I don't claim that all backends will have to work this way.  Just that
> it's a common implementation technique.

Common for some paradigms of development activity, for some
development tasks, and probably also for some programming languages.
It's not universally true by any measure.

> > Until that problem is solved in core, I don't see how a back-end that
> > requires to load a package can be the default.
> 
> Agreed.

But since it was already made the default, it means we must fix this
ASAP.

> >> > That's one way of looking at it.  Another is to say that etags gave
> >> > you more information and thus allowed to find more loose matches,
> >> > which is helpful when your memory is failing you.
> >> M-. jumps to the definition of the "thing under point".  There's no
> >> memory involved.
> > I was talking about "C-u M-."
> 
> C-u M-. also lets you do loose matching, via completion, if your memory
> is failing you.

I don't think completion is the right tool for these searches, because
the name alone doesn't tell enough.  So if we want to base that on
name-completion, we are asking the user to type "something TAB" every
time she needs another candidate.  I don't think this is a very
convenient UI for this job.

> > I didn't say "unfiltered".  I asked for user control of the amount of
> > filtering or of looseness of the matches.  In etags.el terms, think
> > about giving the user control on what goes into find-tag-tag-order.
> 
> That's OK, then.  It seems that etags is simply unable to give
> sufficiently reliable data to filter it effectively, so we're stuck with
> heuristics and user-knobs to choose where they're rather get bitten.

No.  etags provided you a long list, but ordered by importance.  As I
said, I don't remember ever having to examine more than the few first
candidates.  etags relied on this order to save the user from having
to deal with a huge list of potential candidates, at least in most
cases.  etags _can_ effectively filter the results it produces, if you
ask it to use less members in the find-tag-tag-order list.  Or you
could just take the first N candidates of the list it returns, and
ignore the rest.

> > Once again, I question the decision to switch to this UI, when we
> 
> I know, but that's just a waste of time.  The backends won't come before
> this system is standard and common place (and they'll take their time,
> because in the mean time they can keep using their ad-hoc UI which also
> works on older Emacsen).

See above: you assume that the division of functionality between the
UI and the back-ends is at the right place.  I'm not so sure.  If I'm
right, then when more back-ends come, we will see more problems, not
less.

> > It's not!!  There are partial matches shown by completion -- they are
> > the next candidates.
> 
> Again, you choose to look at it this way because you're used to the way
> etags.el worked.

Not to how etags.el worked, to how _I_ worked, what _I_ needed to do
with etags.el's help.  We simply have 2 very different kinds of jobs
to do most of our time.  One more argument for more user control on
the UI level, if you ask me.

> But to me, "find-tag" and "find-tag-tag" are not two different
> matches to my request.  They're just two completely unrelated
> things: either I'm looking for one or I'm looking for the other.

Assuming you know what you are looking for, yes.  I described a
situation that is frequent for me where you generally don't, at least
not well enough to be satisfied by a single exact match.

> The design of xref is based on the hope that the backend can return
> a very small list of candidates, ideally of length 1.

And therein lies its weakness.  I actually don't understand how this
kind of assumption could be allowed to exist, when the _default_
back-end, and one of only 3 existing ones, blows that assumption out
of the water.

> On the contrary, I want a tool that's precise, and gets me directly to
> the only corresponding definition (in the 99% of the cases where
> there's indeed only one) with a single key-press and no questions asked.

Yes, I want that too.  I also want a tool which will write code for
me.  I just don't believe either of these is possible, because in most
of my use cases (which I believe are shared by many in the industry)
no program can ever be lucky enough to guess what definition is that.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Thu, Apr 30 2015, Dmitry Gutov wrote:

> Is there anything else I'm missing that we could do without updating
> existing clients?

next-error calls next-error-find-buffer.  Maybe we could record every
buffer that was returned by next-error-find-buffer and include them in
the candidate list.

Helmut

Re: UI inconveniences with M-.

[Stefan Monnier]

> If the back-end conceals potentially useful information, it should be
> fixed not to do so.

"Conceal" makes it sound like it's ill-intentioned, done on purpose
or something.

For (defadvice find-tag ...), the problem is in advice.el.
Patches welcome, but since this library is on the way out AFAIC, I'd
recommend you don't bother spending time on it.

> That you personally find that information always redundant does not
> mean it's true for everyone else.  There's more than one use case for
> these queries, see below.

We can't know what the user wants and handle all conceivable scenarios.
Maybe the user really wants to jump to this chunk of code somewhere that
does (fset foo bar) and hence ends up overriding the "official"
definition of the function.  But unless you can solve the halting
problem and read the user's mind at the same time, we'll have to settle
for something imperfect.

etags.el also failed miserably in some scenarios which xref/elisp
handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.

> That's your misunderstanding.  I described my user experience from
> using this new feature; I never said where the fixes should be made,

Well, you kept insisting that it's not a question of backend, so maybe
you didn't say where the fixes should go, but you did say where the blame
should go.

> Says you.  Through my naive-user eyes, filtering 140 hits to provide
> just a few is perfectly within the capabilities of the UI, at least in
> principle.

The 140 hits are just a list of locations.  The UI has no fucking clue
whether these are function definitions or what, nor does it know in
which language the file is written.

> IOW, the separation of functionality is in the wrong place.  To be
> useful, some of the "smarts" need to be on the UI side, where user
> control can be best implemented, and where user intent is known much
> better.

The UI has to be agnostic.  So the smarts can't be in the UI.  The API
can be extended to provide the extra smarts that the UI might need, of
course.  E.g. we could add to the API a function that sorts/groups the
entries, so the etags backend can sort them based on "likelyhood" rather
than group them by file.

> I very much hope Emacs will continue to be able to support the kind of
> activities I described above, which AFAIK are very important part of a
> software developer's job throughout the industry.

You fail to understand why complaint about etags.el.  I'm not
complaining about `etags', but about the etags.el front-end, which is in
need of improvement to handle the case where the user is navigating
several completely different projects and doesn't want one to pollute
the other one.

>> I've tried several times to make real use of it, but always found it
>> completely unpalatable.  What with having to build those damn TAGS
>> files, remember to refresh them, remember where they are, constantly
>> tell Emacs again where they are, deal with its inability to find the
>> right spot and having to repeat the "C-u M-." finger gymnastics
>> umpteen times.
> Those are exaggerations.

Partly, yes.  I'm just venting my frustration with the tool, and
pointing out that if xref/elisp (and xref/etags) has some downsides,
etags.el had its own set of downsides.  And some of those shouldn't be
that hard to fix (tho they would probably be a lot easier if we didn't
have to worry about annoying some old-time users because they'd have to
slightly change their habits).

> Building TAGS is almost instant, even in Emacs,

The problem is not computer-time but human-time.

> you need only refresh them very seldom, and Emacs offers the
> place from which to load them so you don't need to remember.

If Emacs knows where the file is, the user shouldn't need to be queried.

> But this, again, is immaterial for this discussion.  I hope you will
> agree that, whatever issues we have with etags, replacing it with
> something that lacks important functionality is not a good idea.

As I said, going back to etags.el is not an option.

> That "little detail" all but invalidates most of my use cases.

Then don't use that backend.  E.g. use xref-etags-mode.

>> C-u M-. also lets you do loose matching, via completion, if your memory
>> is failing you.
> I don't think completion is the right tool for these searches, because
> the name alone doesn't tell enough.

Don't pretend you don't know about xref-apropos.

> See above: you assume that the division of functionality between the
> UI and the back-ends is at the right place.

No I don't.  I don't assume the API is fixed either.  All I assume is
that the UI can't know about the programming language or about the
quality of any given answer, or any such thing.

>> But to me, "find-tag" and "find-tag-tag" are not two different
>> matches to my request.  They're just two completely unrelated
>> things: either I'm looking for one or I'm looking for the other.
> Assuming you know what you are looking for, yes.  I described a
> situation that is frequent for me where you generally don't, at least
> not well enough to be satisfied by a single exact match.

And that's not what M-. for.  For that we have xref-apropos.

> And therein lies its weakness.  I actually don't understand how this
> kind of assumption could be allowed to exist, when the _default_

Because this assumption is known to be obtainable, with help from the
toolchain (the compiler will generally know with 100% certainty the few
possible definitions matching a particular use).


        Stefan

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Fri, 01 May 2015 10:21:04 -0400
> 
> > If the back-end conceals potentially useful information, it should be
> > fixed not to do so.
> 
> "Conceal" makes it sound like it's ill-intentioned, done on purpose
> or something.

Not according to my references.

> > That you personally find that information always redundant does not
> > mean it's true for everyone else.  There's more than one use case for
> > these queries, see below.
> 
> We can't know what the user wants and handle all conceivable scenarios.

Isn't that what I said?

> Maybe the user really wants to jump to this chunk of code somewhere that
> does (fset foo bar) and hence ends up overriding the "official"
> definition of the function.  But unless you can solve the halting
> problem and read the user's mind at the same time, we'll have to settle
> for something imperfect.

Emacs has a school-book solution for these problems: let the user
specify what she wants, with prefix arguments and user options.

My point was that a back-end that isn't prepared for such degree of
control, and decides for the user what she wants is not just
"imperfect", it's semi-broken.

> etags.el also failed miserably in some scenarios which xref/elisp
> handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.

I don't see diff-mode-abrev-table in TAGS, so I don't understand what
you want to demonstrate with that.

> > That's your misunderstanding.  I described my user experience from
> > using this new feature; I never said where the fixes should be made,
> 
> Well, you kept insisting that it's not a question of backend, so maybe
> you didn't say where the fixes should go, but you did say where the blame
> should go.

No, I said that blaming the back-end for what I observed, and saying
that nothing can be done "because that's what the back-end gives" is
not the right way of approaching these kinds of problems.

> > Says you.  Through my naive-user eyes, filtering 140 hits to provide
> > just a few is perfectly within the capabilities of the UI, at least in
> > principle.
> 
> The 140 hits are just a list of locations.  The UI has no fucking clue
> whether these are function definitions or what, nor does it know in
> which language the file is written.

The 140 hits are symbol names, not just locations.  Discerning between
exact matches, caseless matches, and partial matches does not need any
language-specific knowledge.  I don't understand what problems you see
here.

But if you are certain that the fixes belong to the back-end, it's
fine with me.  Just don't tell me that the back-end is to blame, and
therefore there's nothing that can be done.

> > IOW, the separation of functionality is in the wrong place.  To be
> > useful, some of the "smarts" need to be on the UI side, where user
> > control can be best implemented, and where user intent is known much
> > better.
> 
> The UI has to be agnostic.  So the smarts can't be in the UI.

The question is which smarts.  I'm saying that the UI cannot blindly
trust the back-end to always DTRT, it needs to control it according to
its needs and user preferences.  These smarts _must_ be in the UI.
Having just 2 or 3 query types -- either get me only one match or give
me all of them -- is too rough, there must be more shades of gray, and
more ways for the user to extend and refine the query.

The old API provided an approximation to this by returning a long list
in order of importance, so where you stopped asking for more was a
rather crude, but usually effective way of getting exactly what you
wanted.  This is now gone, so we must find other, hopefully better
ways of giving the user the same fine-grained control on what she gets
in response to a query.

> > I very much hope Emacs will continue to be able to support the kind of
> > activities I described above, which AFAIK are very important part of a
> > software developer's job throughout the industry.
> 
> You fail to understand why complaint about etags.el.  I'm not
> complaining about `etags', but about the etags.el front-end, which is in
> need of improvement to handle the case where the user is navigating
> several completely different projects and doesn't want one to pollute
> the other one.

This issue is tangential to the subject f this discussion.  My
complaints are valid even if you work with a single project, and in
fact were observed while working with a single project.

> >> I've tried several times to make real use of it, but always found it
> >> completely unpalatable.  What with having to build those damn TAGS
> >> files, remember to refresh them, remember where they are, constantly
> >> tell Emacs again where they are, deal with its inability to find the
> >> right spot and having to repeat the "C-u M-." finger gymnastics
> >> umpteen times.
> > Those are exaggerations.
> 
> Partly, yes.  I'm just venting my frustration with the tool, and
> pointing out that if xref/elisp (and xref/etags) has some downsides,
> etags.el had its own set of downsides.

Irrelevant.  I wasn't saying etags was better, or didn't have
disadvantages.  I was talking about deficiencies of the current
solution, as a whole, for looking up definitions of symbols.  I don't
really care which back-end is used for that, but I do care about the
results, and I do care when if I switch back-ends and suddenly get
very different results.

> (tho they would probably be a lot easier if we didn't have to worry
> about annoying some old-time users because they'd have to slightly
> change their habits).

Yes, I'm an annoying guy.  Feel free to ask me to step down and
disappear from here, if that annoys you too much.  (The profanities
are already a sign of that.)

> > Building TAGS is almost instant, even in Emacs,
> 
> The problem is not computer-time but human-time.

What human time?  The time it takes to type "make TAGS"?

> > you need only refresh them very seldom, and Emacs offers the
> > place from which to load them so you don't need to remember.
> 
> If Emacs knows where the file is, the user shouldn't need to be queried.

Emacs can guess where the user _might_ want it, but it has to allow
for seldom exceptions, otherwise how can the user switch to another
project or add its data to the current one?

> > But this, again, is immaterial for this discussion.  I hope you will
> > agree that, whatever issues we have with etags, replacing it with
> > something that lacks important functionality is not a good idea.
> 
> As I said, going back to etags.el is not an option.

It's definitely an option for this curmudgeon, if the new-and-improved
solution will not become better.

> > That "little detail" all but invalidates most of my use cases.
> 
> Then don't use that backend.  E.g. use xref-etags-mode.

Are we again talking about my personal problems?  If they are only
mine, I can solve them very easily, and don't need this discussion or
the bug reports to do so.

What do you tell users whose use case is similar to mine?  "Use
xref-etags-mode"?  Then why did we switch away of etags.el, if we
still need to use its core machinery?

> >> C-u M-. also lets you do loose matching, via completion, if your memory
> >> is failing you.
> > I don't think completion is the right tool for these searches, because
> > the name alone doesn't tell enough.
> 
> Don't pretend you don't know about xref-apropos.

I don't want 2 different commands.  I know about etags-apropos as
well, but never used it, because I never needed to.  Forcing me to use
2 different commands where I could use one is an annoyance.

Anyway, I'm out of this discussion.  Good luck selling your ideas and
use cases to others.  I'm not sold.  And I don't enjoy having to read
uncalled-for profanities in response to legitimate questions and
issues.

Re: UI inconveniences with M-.

[Stefan Monnier]

>> etags.el also failed miserably in some scenarios which xref/elisp
>> handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.
> I don't see diff-mode-abrev-table in TAGS, so I don't understand what
> you want to demonstrate with that.

Exactly that: etags fails to find the definition site of
diff-mode-abrev-table, whereas xref/elisp does find it.

>> As I said, going back to etags.el is not an option.
> It's definitely an option for this curmudgeon, if the new-and-improved
> solution will not become better.

Of course you can.  I'm only talking about the default.

> What do you tell users whose use case is similar to mine?  "Use
> xref-etags-mode"?  Then why did we switch away of etags.el, if we
> still need to use its core machinery?

We do want to support the etags backend for a long time to come.
The old etags.el UI on the other hand, will most likely be phased out
(or rewritten on top of the new API no it can also be used with other
backends).

> I don't want 2 different commands.  I know about etags-apropos as
> well, but never used it, because I never needed to.  Forcing me to use
> 2 different commands where I could use one is an annoyance.

It's a tradeoff.  The upside is that M-. gets you there much quicker
then M-. RET C-u M-. C-u M-., and that xref-apropos gives you a quick
overview of all the matches, compared to cycling through C-u M-. where
you never know how long the cycle will be nor where it will take you next.

> Anyway, I'm out of this discussion.  Good luck selling your ideas and
> use cases to others.  I'm not sold.  And I don't enjoy having to read
> uncalled-for profanities in response to legitimate questions and
> issues.

I've found you didn't have many constructive suggestions and spent too
much time complaining about how the new system doesn't work like the
old one.
That's frustrating.


        Stefan

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 04/30/2015 04:42 PM, Eli Zaretskii wrote:

> While etags shows a very long list of potential matches, it produces
> them in the order of importance; I don't remember ever having to
> examine more than the first few hits (and thus wasn't even aware of of
> the sheer length of the full list).  So with the old UI the
> ridiculously long list of candidates was never in my face.  The new
> UI, by contrast, makes this acutely evident by showing me the full
> list, and on top of that it destroys the order produced by the
> back-end, which would allow me to look only at the few first hits.

That is unfortunate indeed. Browsing the xref-find-apropos results 
segregated by the match distance might prove useful.

> If I wanted to talk about the code, I'd say something like
> "this-and-that function does wrong things because of so-and-so".

I think both I and Stefan very much wanted you to look at the actual 
code here. It's much easier to make demands on functionality than to 
propose a clean and modular design.

> Through my naive-user eyes, filtering 140 hits to provide
> just a few is perfectly within the capabilities of the UI, at least in
> principle.

That is, of course, possible. But then we'll need to define some 
universal language-arnostic metadata that the UI would use to operate.

> It sounds more and more like it could be the fault of the design, and
> specifically of how functionality is divided between the back-end and
> the UI.  Let me elaborate.
>
> <...>
> To me, this means that separating the back-ends from the UI while
> leaving the UI "dumb" is basically unworkable, because such a
> separation does not really separate anything: there will still be a
> very high degree of coherency and cohesion between the two parts.  Or
> maybe there will only be one usable back-end, and the rest will
> bit-rot.

Sorry, all I see here is a lot of conjecture. The current 
implementations took not a lot of code, and they work well enough.

You seem to argue in favor of extracting some common code into the UI, 
but there's not much of it that I can see. Again, it would help if your 
proposals were more technical and lower-level.

> Likewise, people who mainly work with small projects, where the code
> line count is in hundreds or a small number of thousands, might never
> need the less-than-exact matches provided by etags: Grep is good
> enough and fast enough for that.  But you cannot do without etags in a
> larger project.  (You also cannot do without being able to find all
> references to an identifier, which is why I'm using ID Utils for the
> past 15 years, and was disappointed to learn that xref-find-references
> has no back-ends that implemented this kind of search.)

M-x xref-find-references is implemented now, for both backends, using 
Grep or one of the faster tools (semantic/symref makes the choice), and 
the "symbol bounds" matching. Hope you're happier now.

By the way, CEDET has had a similar feature for a while (try `M-x 
semantic-mode', then `C-c , G' on some function, in a C file). Arguably, 
even with a better interface.

Any reason why you haven't been using it?

> That "little detail" all but invalidates most of my use cases.  We
> seem to have very different development experiences, probably due to
> what each one of us does most of the time, but I hope you can see that
> there are valid use cases outside of your personal experience, and
> Emacs should support them if it wants to be a good IDE.

I'm sure we want it to be. However, I've seen no indication that the 
majority of users prefers find-tag over find-function. Maybe some hard 
data would change that.

> And I'm not even sure your ideas of how to solve that "little detail"
> are workable, because of the potentially adverse consequences of
> loading code you don't actually need (or want) to execute.  What if
> the code is buggy, or dangerous, or simply does things you don't want
> to be done in your Emacs session?

That's a valid concern, but you'd have to read that kind of project from 
top to bottom first. Then you can load it and proceed to use the 
navigation functions.

In any case, we should probably return to this example after it occurs 
in our ecosystem at least once.

> I don't know what that means, and don't know enough about JDEE to talk
> about it.  In any case, Java is not a "classic" compiled language, as
> a Jave development environment is generally capable of running the
> code in an interpreter.

There are several ones for C, example: 
http://www.drdobbs.com/cpp/ch-a-cc-interpreter-for-script-computing/184402054

Not that it has any relevance to the xref backend interface.

> But since it was already made the default, it means we must fix this
> ASAP.

-1.

> I don't think completion is the right tool for these searches, because
> the name alone doesn't tell enough.  So if we want to base that on
> name-completion, we are asking the user to type "something TAB" every
> time she needs another candidate.  I don't think this is a very
> convenient UI for this job.

Indeed, xref-find-apropos is better for that.

> See above: you assume that the division of functionality between the
> UI and the back-ends is at the right place.  I'm not so sure.  If I'm
> right, then when more back-ends come, we will see more problems, not
> less.

That's conjecture again.

> Assuming you know what you are looking for, yes.  I described a
> situation that is frequent for me where you generally don't, at least
> not well enough to be satisfied by a single exact match.

I'd like to repeat my suggestion:

- You can bind `M-.' to `xref-find-apropos' in your config.
- We can improve its output somehow.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 12:04 AM, Stefan Monnier wrote:

> then M-. RET C-u M-. C-u M-., and that xref-apropos gives you a quick

You seem to be subtly suggesting that the command should be renamed.

Re: UI inconveniences with M-.

[Stefan Monnier]

[ Moving this to emacs-devel.  ]

> Maybe the solution is to define the ability for a backend to return groups,
> probably nested ones.

I don't see much need for nesting.  I think the issue is mostly one of
sorting in *xref* (or in the cycling, if we were to use a hypothetical
other UI that provides functionality like the old etags.el).

So, let's focus on this.
My suggestion would be either:
- make xref-find-function return not just a list of locations, but also
  an indication whether they're pre-sorted or not.
- make xref-find-function return not a list of locations, but a list of
  lists of locations (i.e. every location inside a sub-list is
  considered to have the same "level/quality/likelihood", but the lists
  are sorted by their "level/quality/likelihood").
- let the backend provide its own sorting function to override the
  current "group by file" sorting.

Basically, I think the focus should be on making sure that we can
re-implement the old behavior on top of the new API when using the
xref/etags backend.

Another thing that we should consider is how to let the user choose
which backend to use.  Currently we only have xref-etags-mode, but
I think it would make sense to make it possible for the user to
dynamically choose among the potentially many more backends (e.g. we
could add an xref/grep backend, and there will hopefulyl be more using
things like id-utils, gtags, ...) and maybe also to use several backends
at the same time (merging the results).  I don't have any suggestions
for what this could/should look like, OTOH.  But maybe one way to do
that would be to let each backend expose via the API a set of knobs
(one of which would be an "enable/disable" knob), and then provide an
xref command that can manage these knobs conveniently.


        Stefan

Re: UI inconveniences with M-.

[Stefan Monnier]

>> then M-. RET C-u M-. C-u M-., and that xref-apropos gives you a quick
> You seem to be subtly suggesting that the command should be renamed.

No, I'm just lazy,


        Stefan

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dgutov@yandex.ru,  emacs-devel@gnu.org
> Date: Fri, 01 May 2015 17:04:48 -0400
> 
> > Anyway, I'm out of this discussion.  Good luck selling your ideas and
> > use cases to others.  I'm not sold.  And I don't enjoy having to read
> > uncalled-for profanities in response to legitimate questions and
> > issues.
> 
> I've found you didn't have many constructive suggestions and spent too
> much time complaining about how the new system doesn't work like the
> old one.
> That's frustrating.

Please find a more civilized way of expressing your frustration.

Re: UI inconveniences with M-.

[Helmut Eller]

On Sat, May 02 2015, Stefan Monnier wrote:

> - let the backend provide its own sorting function to override the
>   current "group by file" sorting.

The current grouping preserves the order: xrefs are grouped by the
string returned by xref-location-group and both the order of the groups
and the order of xrefs within the group is preserved. See
xref--alistify.

> Basically, I think the focus should be on making sure that we can
> re-implement the old behavior on top of the new API when using the
> xref/etags backend.

Theoretically this can already be done by ignoring xref-location-group
or by specializing xref-location-group for xref-etags-location so that it
returns always the same group.

Thinking about it: for etags it would make sense to group xrefs by name
of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
"globally"; xrefs are reordered only within a TAGS file.

Helmut

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 00:11:28 +0300
> 
> > If I wanted to talk about the code, I'd say something like
> > "this-and-that function does wrong things because of so-and-so".
> 
> I think both I and Stefan very much wanted you to look at the actual 
> code here. It's much easier to make demands on functionality than to 
> propose a clean and modular design.

One person can only do so much, given her free time, and can only be
an expert in so many fields.  When you or Stefan report a problem with
the display engine, or in some other area where I know enough, I don't
ask for a design before I start working on it.  In this case, all I
have to offer is user experience, some requirements for what I
consider to be a good solution, and some general guidelines for such a
solution (which I only provided in response to repeated demands to do
so).  If that is not useful, perhaps we should revise our instructions
for bug reporting.

IOW, I was reporting problems in an area where I know very little.  I
don't think it's fair to demand that I provide, let alone code, a
solution, as a prerequisite for acting on my report.  I think the job
of making the feature better in response to bug reports is the job of
those who worked on the feature to begin with, and thus have intimate
knowledge of the design and the implementation.

> > Through my naive-user eyes, filtering 140 hits to provide
> > just a few is perfectly within the capabilities of the UI, at least in
> > principle.
> 
> That is, of course, possible. But then we'll need to define some 
> universal language-arnostic metadata that the UI would use to operate.

I think this kind of universal metadata is very much clear and mostly
already supported.  After all, Emacs has been doing this kind of jobs
for a very long time, which is why we have notions like e.g. "symbols",
which each language defines differently.

> > It sounds more and more like it could be the fault of the design, and
> > specifically of how functionality is divided between the back-end and
> > the UI.  Let me elaborate.
> >
> > <...>
> > To me, this means that separating the back-ends from the UI while
> > leaving the UI "dumb" is basically unworkable, because such a
> > separation does not really separate anything: there will still be a
> > very high degree of coherency and cohesion between the two parts.  Or
> > maybe there will only be one usable back-end, and the rest will
> > bit-rot.
> 
> Sorry, all I see here is a lot of conjecture.

Of course it is.  What else did you expect from a bystander who wasn't
involved in the design and implementation?

It is up to you to do whatever you want with this conjecture.  Some
people pay a lot of money to get my conjectures in this and similar
fields.  You get it for free.

> The current implementations took not a lot of code, and they work
> well enough.

That's not an evidence of the design validity, not IME.  This feature
is with us for too short time to be able to draw conclusions about its
design.  At least it "didn't work well enough" for me, which is a sign
that it isn't perfect.  And you already made quite a few changes based
on my experience.  I think it might be a good time to step back and
review those changes, looking for some more fundamental issues that
might benefit from some redesign.  I don't know what you will find, if
you do that, but I do know that if you continue to insist that the
design is perfect, you will never see its flaws, if they exist.

> By the way, CEDET has had a similar feature for a while (try `M-x 
> semantic-mode', then `C-c , G' on some function, in a C file). Arguably, 
> even with a better interface.
> 
> Any reason why you haven't been using it?

Partly because CEDET is too heavyweight for most of my needs, and
partly because I simply didn't have enough time to learn it.

> > And I'm not even sure your ideas of how to solve that "little detail"
> > are workable, because of the potentially adverse consequences of
> > loading code you don't actually need (or want) to execute.  What if
> > the code is buggy, or dangerous, or simply does things you don't want
> > to be done in your Emacs session?
> 
> That's a valid concern, but you'd have to read that kind of project from 
> top to bottom first. Then you can load it and proceed to use the 
> navigation functions.

That's impossible.  I'm talking about projects whose line counts are
in hundreds of thousands, sometimes millions.  You cannot read such
project from top to bottom, when all you need to do is fix some bug or
find the reason for some particular behavior: you will never make your
deadline.  Using the tools I'm talking about is the only way to find
the _relevant_ places which you do need to read and understand.

IOW, your methodology would put the cart before the horse, in these
use cases.

> > I don't know what that means, and don't know enough about JDEE to talk
> > about it.  In any case, Java is not a "classic" compiled language, as
> > a Jave development environment is generally capable of running the
> > code in an interpreter.
> 
> There are several ones for C, example: 
> http://www.drdobbs.com/cpp/ch-a-cc-interpreter-for-script-computing/184402054
> 
> Not that it has any relevance to the xref backend interface.

Then why bring it up?

> > See above: you assume that the division of functionality between the
> > UI and the back-ends is at the right place.  I'm not so sure.  If I'm
> > right, then when more back-ends come, we will see more problems, not
> > less.
> 
> That's conjecture again.

And I have some gray heir to back it up with.  I have learned from
long experience that good design is frequently backed up by intuition
and "conjecture".  I'm not necessarily saying I'm right in this case,
but what if I am?  Shouldn't you at least consider that, instead of
rejecting it flatly as "conjecture" and sticking to the original
design as if it were a scripture?

Re: UI inconveniences with M-.

[Fumitaka Tokumitsu]

[-- Attachment #1: Type: text/plain, Size: 7848 bytes --]

さほと



—
Sent from Mailbox

On Wed, Apr 29, 2015 at 11:41 AM, Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> [ Moving this to emacs-devel.  ]
>>> > Emacs 24 also had "C-u M-." to go to the next one.  This one doesn't;
>>> > moreover, if you try "C-u M-.", you get prompted for the symbol again,
>>> > and if you type the same one, you get nowhere.  The other matches are
>>> > only available via completion, see below.
>>> Maybe we should write an xref-old-ui-mode.
>> I didn't ask for the old UI.  (If I want it, I can still have it,
>> since the etags.el commands are still available.)  I'm okay with the
>> new UI.  This bug report is about the new UI, and its goal is to make
>> the new UI better.
> Then I don't understand what you mean by the text I quoted.  E.g.:
>    if you try "C-u M-.", you get prompted for the symbol again,
>    and if you type the same one, you get nowhere
> Well, duh!  What did you expect?  In the new UI, "C-u M-." jumps to
> *the* definition (the "C-u" is only used to let you type the name of
> thing you're looking for), so of course if you just jumped to it, asking
> to jump to it again will get you nowhere else.  I guess it would be OK
> to make M-. emit a message like "Hello?  We're already here!" in this
> corner case, if you want to make sure something happens.
>> old UI was consistent and complete: it always displayed the first/next
>> match and provided a command to go to the next/previous one.  By
>> contrast, the new UI is inconsistent: with some back-ends it shows the
>> list of matches and allows to navigate it, with others it shows only
>> the first match and does not give any way to get the next match.
> In the "M-. find-tag" example, the UI does let you see all matches.
> It's just that there's only one.  If you want, we could make this case
> popup an *xref* buffer with a single entry, but I think it makes more
> sense to just jump straight to that one entry instead.
>> But that's largely immaterial: this bug report is not about the
>> back-end, it's about the UI.
> I wouldn't be so sure.
>>> Arguably, the find-tag advice in org-ctags.el could be offered as well,
>>> if org-ctags.el happens to be loaded
>> ??? Why should the xref matches depend on what is and isn't loaded?
> Because that's how xref/elisp works.
>> That would make xref exactly equivalent to "M-x apropos", which means
>> this mode of operation would make xref entirely redundant.
> To you, maybe.  Personally, I find it's *much* quicker to use M-. than
> to use C-h f RET <go-to-the-link> RET, and it has the added benefit that
> M-, brings me right back if I need to.
>> To say nothing of the fact that this doesn't scale to any language
>> except ELisp.
> Not at all.  Many (most?) packages which offer a functionality along the
> lines of "M-. to jump to the definition" use an implementation technique
> which is fundamentally similar/equivalent (obviously, they don't query
> a running Emacs, but they query a running REPL).
>> We could offer this mode as an optional feature, but it certainly
>> shouldn't be the default.
> Many users here disagree.
>> (One of my worst annoyances is to type a C-h command and be presented
>> with "[No match]", because some package is not loaded or some function
>> is not available in the Emacs configuration I happen to be using.)
> Yes, that's a long standing problem, indeed.  I have a local hack which
> tries to add to loaddefs.el a concise description of where definitions
> might be found (basically, for each file, list the few prefixes used in
> that file), so that "C-h f" can load those packages when needed.
> It's "proof of concept" and it has occasionally rendered my Emacs
> session unusable (e.g. because it ended up deciding to try and load,
> say, w32-fns.el to see if it might contain the definition).  I think
> I've now solved most of those problems in the w32-* files, but the
> underlying risk is still there.
>> That's one way of looking at it.  Another is to say that etags gave
>> you more information and thus allowed to find more loose matches,
>> which is helpful when your memory is failing you.
> M-. jumps to the definition of the "thing under point".  There's no
> memory involved.
> If you're not sure what you're looking for, then you're expected to use
> the completion facilities in C-u M-.
>> But in the context of this bug report, that, too, is immaterial: if we
>> think the etags back-end gives too much information, by all means
>> let's filter it before presenting the matches.
> Yes, that's clearly very much needed.
>> Bonus points for making the filtering optional, since some people
>> might like that, and in some situations even you might need it.
> I disagree.  The filtering is needed by the design of the xref API.
> If you want to see the "unfiltered" data, then use M-x xref-find-apropos.
>> Then let's fix the results we display with the etags back-end to show
>> only the relevant ones.
> Yes, please.
>> Saying that the back-end returns a confusingly large amount of bogus
>> matches, and then switching to a UI that assumes much smarter
>> back-ends (which don't really exist) makes very little sense to me.
> The smarter backends already exist, in so far as there is already code out
> there which provides the needed functionality.  It just doesn't use the
> xref API yet, because it was written before xref.el.
>> The old code attempted to support both use cases, by showing the exact
>> match(es) first, followed by less likely ones.  We do similar things
>> in other commands.  The advantages of such a method are that (1) you
>> don't need to second-guess the user, (2) you provide only one command
>> for the user to remember, and (3) if the user only _thinks_ she knows
>> the name, but really doesn't, she can find the information without
>> having to invoke another command.
>>> I think that's an important feature of the new code in this respect.
>> But it is incomplete without a means to get to the other possible
>> matches in this case.
>> To summarize my points in this sub-thread:
>>   . the UI should depend much less on the back-end, ideally not at all
> You keep repeating this, but that is absolutely meaningless to me (kind
> of like "it doesn't work" in bug reports):
> - This consistency was trivially obtained with the old etags.el code
>   since it had one 1 backend.
> - What the fuck should the new UI do if one backend returns 1 match and
>   another returns a hundred (as in your "find-tag" example)?
>>   . there should be a way to go to the next/previous match if the
>>     *xref* buffer is not displayed (or not created in the first
>>     place); if this happens because there's only one match, we should
>>     say so explicitly
> The only case where the *xref* is not displayed is when there's only
> 1 match returned by the backend.  So "go to the next/previous match" is
> again meaningless.
>>   . when there are more than one possible match, the UI should behave
>>     similarly, i.e. display the *xref* buffer; when there's only one
>>     match, it should _never_ display *xref*,
> That's exactly what the code does.
>>     and should display an indication of the fact that there's only one
>>     match
> The current code "displays" this indication by jumping to the sole match
> instead of jumping to the *xref* buffer.  I think it's clear enough.
>>   . if the criteria for filtering of the matches are very different
>>     between the possible back-ends, there should be some agreed-upon
>>     uniform default that returns roughly the same number of matches
>>     with all back-ends, and the rest should be controlled by user
>>     options
> I don't see what that could concretely look like.
>         Stefan

[-- Attachment #2: Type: text/html, Size: 8951 bytes --]

Re: UI inconveniences with M-.

[Fumitaka Tokumitsu]

[-- Attachment #1: Type: text/plain, Size: 5883 bytes --]

にた泣かねけあせけかきねはきかきか



—
Sent from Mailbox

On Fri, May 1, 2015 at 11:21 PM, Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

>> If the back-end conceals potentially useful information, it should be
>> fixed not to do so.
> "Conceal" makes it sound like it's ill-intentioned, done on purpose
> or something.
> For (defadvice find-tag ...), the problem is in advice.el.
> Patches welcome, but since this library is on the way out AFAIC, I'd
> recommend you don't bother spending time on it.
>> That you personally find that information always redundant does not
>> mean it's true for everyone else.  There's more than one use case for
>> these queries, see below.
> We can't know what the user wants and handle all conceivable scenarios.
> Maybe the user really wants to jump to this chunk of code somewhere that
> does (fset foo bar) and hence ends up overriding the "official"
> definition of the function.  But unless you can solve the halting
> problem and read the user's mind at the same time, we'll have to settle
> for something imperfect.
> etags.el also failed miserably in some scenarios which xref/elisp
> handles acceptably.  E.g. try C-u M-. diff-mode-abrev-table RET.
>> That's your misunderstanding.  I described my user experience from
>> using this new feature; I never said where the fixes should be made,
> Well, you kept insisting that it's not a question of backend, so maybe
> you didn't say where the fixes should go, but you did say where the blame
> should go.
>> Says you.  Through my naive-user eyes, filtering 140 hits to provide
>> just a few is perfectly within the capabilities of the UI, at least in
>> principle.
> The 140 hits are just a list of locations.  The UI has no fucking clue
> whether these are function definitions or what, nor does it know in
> which language the file is written.
>> IOW, the separation of functionality is in the wrong place.  To be
>> useful, some of the "smarts" need to be on the UI side, where user
>> control can be best implemented, and where user intent is known much
>> better.
> The UI has to be agnostic.  So the smarts can't be in the UI.  The API
> can be extended to provide the extra smarts that the UI might need, of
> course.  E.g. we could add to the API a function that sorts/groups the
> entries, so the etags backend can sort them based on "likelyhood" rather
> than group them by file.
>> I very much hope Emacs will continue to be able to support the kind of
>> activities I described above, which AFAIK are very important part of a
>> software developer's job throughout the industry.
> You fail to understand why complaint about etags.el.  I'm not
> complaining about `etags', but about the etags.el front-end, which is in
> need of improvement to handle the case where the user is navigating
> several completely different projects and doesn't want one to pollute
> the other one.
>>> I've tried several times to make real use of it, but always found it
>>> completely unpalatable.  What with having to build those damn TAGS
>>> files, remember to refresh them, remember where they are, constantly
>>> tell Emacs again where they are, deal with its inability to find the
>>> right spot and having to repeat the "C-u M-." finger gymnastics
>>> umpteen times.
>> Those are exaggerations.
> Partly, yes.  I'm just venting my frustration with the tool, and
> pointing out that if xref/elisp (and xref/etags) has some downsides,
> etags.el had its own set of downsides.  And some of those shouldn't be
> that hard to fix (tho they would probably be a lot easier if we didn't
> have to worry about annoying some old-time users because they'd have to
> slightly change their habits).
>> Building TAGS is almost instant, even in Emacs,
> The problem is not computer-time but human-time.
>> you need only refresh them very seldom, and Emacs offers the
>> place from which to load them so you don't need to remember.
> If Emacs knows where the file is, the user shouldn't need to be queried.
>> But this, again, is immaterial for this discussion.  I hope you will
>> agree that, whatever issues we have with etags, replacing it with
>> something that lacks important functionality is not a good idea.
> As I said, going back to etags.el is not an option.
>> That "little detail" all but invalidates most of my use cases.
> Then don't use that backend.  E.g. use xref-etags-mode.
>>> C-u M-. also lets you do loose matching, via completion, if your memory
>>> is failing you.
>> I don't think completion is the right tool for these searches, because
>> the name alone doesn't tell enough.
> Don't pretend you don't know about xref-apropos.
>> See above: you assume that the division of functionality between the
>> UI and the back-ends is at the right place.
> No I don't.  I don't assume the API is fixed either.  All I assume is
> that the UI can't know about the programming language or about the
> quality of any given answer, or any such thing.
>>> But to me, "find-tag" and "find-tag-tag" are not two different
>>> matches to my request.  They're just two completely unrelated
>>> things: either I'm looking for one or I'm looking for the other.
>> Assuming you know what you are looking for, yes.  I described a
>> situation that is frequent for me where you generally don't, at least
>> not well enough to be satisfied by a single exact match.
> And that's not what M-. for.  For that we have xref-apropos.
>> And therein lies its weakness.  I actually don't understand how this
>> kind of assumption could be allowed to exist, when the _default_
> Because this assumption is known to be obtainable, with help from the
> toolchain (the compiler will generally know with 100% certainty the few
> possible definitions matching a particular use).
>         Stefan

[-- Attachment #2: Type: text/html, Size: 6781 bytes --]

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Date: Sat, 02 May 2015 09:59:47 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Thinking about it: for etags it would make sense to group xrefs by name
> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
> "globally"; xrefs are reordered only within a TAGS file.

What if the different TAGS files actually describe the same project,
like lisp/TAGS and src/TAGS in the Emacs case?

Re: UI inconveniences with M-.

[Fumitaka Tokumitsu]

[-- Attachment #1: Type: text/plain, Size: 6401 bytes --]

こなた 

やなまかわ

  おたかたけ 

ま



—
Sent from Mailbox

On Sat, May 2, 2015 at 5:14 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> Cc: emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sat, 2 May 2015 00:11:28 +0300
>> 
>> > If I wanted to talk about the code, I'd say something like
>> > "this-and-that function does wrong things because of so-and-so".
>> 
>> I think both I and Stefan very much wanted you to look at the actual 
>> code here. It's much easier to make demands on functionality than to 
>> propose a clean and modular design.
> One person can only do so much, given her free time, and can only be
> an expert in so many fields.  When you or Stefan report a problem with
> the display engine, or in some other area where I know enough, I don't
> ask for a design before I start working on it.  In this case, all I
> have to offer is user experience, some requirements for what I
> consider to be a good solution, and some general guidelines for such a
> solution (which I only provided in response to repeated demands to do
> so).  If that is not useful, perhaps we should revise our instructions
> for bug reporting.
> IOW, I was reporting problems in an area where I know very little.  I
> don't think it's fair to demand that I provide, let alone code, a
> solution, as a prerequisite for acting on my report.  I think the job
> of making the feature better in response to bug reports is the job of
> those who worked on the feature to begin with, and thus have intimate
> knowledge of the design and the implementation.
>> > Through my naive-user eyes, filtering 140 hits to provide
>> > just a few is perfectly within the capabilities of the UI, at least in
>> > principle.
>> 
>> That is, of course, possible. But then we'll need to define some 
>> universal language-arnostic metadata that the UI would use to operate.
> I think this kind of universal metadata is very much clear and mostly
> already supported.  After all, Emacs has been doing this kind of jobs
> for a very long time, which is why we have notions like e.g. "symbols",
> which each language defines differently.
>> > It sounds more and more like it could be the fault of the design, and
>> > specifically of how functionality is divided between the back-end and
>> > the UI.  Let me elaborate.
>> >
>> > <...>
>> > To me, this means that separating the back-ends from the UI while
>> > leaving the UI "dumb" is basically unworkable, because such a
>> > separation does not really separate anything: there will still be a
>> > very high degree of coherency and cohesion between the two parts.  Or
>> > maybe there will only be one usable back-end, and the rest will
>> > bit-rot.
>> 
>> Sorry, all I see here is a lot of conjecture.
> Of course it is.  What else did you expect from a bystander who wasn't
> involved in the design and implementation?
> It is up to you to do whatever you want with this conjecture.  Some
> people pay a lot of money to get my conjectures in this and similar
> fields.  You get it for free.
>> The current implementations took not a lot of code, and they work
>> well enough.
> That's not an evidence of the design validity, not IME.  This feature
> is with us for too short time to be able to draw conclusions about its
> design.  At least it "didn't work well enough" for me, which is a sign
> that it isn't perfect.  And you already made quite a few changes based
> on my experience.  I think it might be a good time to step back and
> review those changes, looking for some more fundamental issues that
> might benefit from some redesign.  I don't know what you will find, if
> you do that, but I do know that if you continue to insist that the
> design is perfect, you will never see its flaws, if they exist.
>> By the way, CEDET has had a similar feature for a while (try `M-x 
>> semantic-mode', then `C-c , G' on some function, in a C file). Arguably, 
>> even with a better interface.
>> 
>> Any reason why you haven't been using it?
> Partly because CEDET is too heavyweight for most of my needs, and
> partly because I simply didn't have enough time to learn it.
>> > And I'm not even sure your ideas of how to solve that "little detail"
>> > are workable, because of the potentially adverse consequences of
>> > loading code you don't actually need (or want) to execute.  What if
>> > the code is buggy, or dangerous, or simply does things you don't want
>> > to be done in your Emacs session?
>> 
>> That's a valid concern, but you'd have to read that kind of project from 
>> top to bottom first. Then you can load it and proceed to use the 
>> navigation functions.
> That's impossible.  I'm talking about projects whose line counts are
> in hundreds of thousands, sometimes millions.  You cannot read such
> project from top to bottom, when all you need to do is fix some bug or
> find the reason for some particular behavior: you will never make your
> deadline.  Using the tools I'm talking about is the only way to find
> the _relevant_ places which you do need to read and understand.
> IOW, your methodology would put the cart before the horse, in these
> use cases.
>> > I don't know what that means, and don't know enough about JDEE to talk
>> > about it.  In any case, Java is not a "classic" compiled language, as
>> > a Jave development environment is generally capable of running the
>> > code in an interpreter.
>> 
>> There are several ones for C, example: 
>> http://www.drdobbs.com/cpp/ch-a-cc-interpreter-for-script-computing/184402054
>> 
>> Not that it has any relevance to the xref backend interface.
> Then why bring it up?
>> > See above: you assume that the division of functionality between the
>> > UI and the back-ends is at the right place.  I'm not so sure.  If I'm
>> > right, then when more back-ends come, we will see more problems, not
>> > less.
>> 
>> That's conjecture again.
> And I have some gray heir to back it up with.  I have learned from
> long experience that good design is frequently backed up by intuition
> and "conjecture".  I'm not necessarily saying I'm right in this case,
> but what if I am?  Shouldn't you at least consider that, instead of
> rejecting it flatly as "conjecture" and sticking to the original
> design as if it were a scripture?

[-- Attachment #2: Type: text/html, Size: 7417 bytes --]

Re: UI inconveniences with M-.

[Helmut Eller]

On Sat, May 02 2015, Eli Zaretskii wrote:

>> Thinking about it: for etags it would make sense to group xrefs by name
>> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
>> "globally"; xrefs are reordered only within a TAGS file.
>
> What if the different TAGS files actually describe the same project,
> like lisp/TAGS and src/TAGS in the Emacs case?

Always the same: visit-tags-table-buffer returns the TAGS files in a
certain order; say src/TAGS before lisp/TAGS.  Within a TAGS file
find-tag-tag-order or etags-xref-find-definitions-tag-order is used for
sorting but in the final result xrefs from src/TAGS always stay before
those from lisp/TAGS.

I think that's what find-tag-in-order does too, but I'm not 100% sure
because find-tag-in-order and visit-tags-table-buffer are very
complicated.

Helmut

Re: UI inconveniences with M-.

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Cc: emacs-devel@gnu.org,  monnier@iro.umontreal.ca
> Date: Sat, 02 May 2015 11:09:23 +0200
> 
> On Sat, May 02 2015, Eli Zaretskii wrote:
> 
> >> Thinking about it: for etags it would make sense to group xrefs by name
> >> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
> >> "globally"; xrefs are reordered only within a TAGS file.
> >
> > What if the different TAGS files actually describe the same project,
> > like lisp/TAGS and src/TAGS in the Emacs case?
> 
> Always the same: visit-tags-table-buffer returns the TAGS files in a
> certain order; say src/TAGS before lisp/TAGS.  Within a TAGS file
> find-tag-tag-order or etags-xref-find-definitions-tag-order is used for
> sorting but in the final result xrefs from src/TAGS always stay before
> those from lisp/TAGS.

I was asking whether grouping by TAGS file is what the user might want
in this case, since, for example in Emacs, the distinction between
Lisp primitives implemented in C (and thus mentioned in src/TAGS), and
subroutines written in Lisp, does not necessarily exist.  Would the
users mind that, say, forward-line is separated from forward-paragraph?

I don't know the answer to that question, but I think it is worth
considering.

Re: UI inconveniences with M-.

[Helmut Eller]

On Sat, May 02 2015, Eli Zaretskii wrote:

> Would the
> users mind that, say, forward-line is separated from forward-paragraph?

It seems the me that etags users were very satisfied with the order in
which find-tag returned the results.

Helmut

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 11:12 AM, Eli Zaretskii wrote:

> One person can only do so much, given her free time, and can only be
> an expert in so many fields.  When you or Stefan report a problem with
> the display engine, or in some other area where I know enough, I don't
> ask for a design before I start working on it.

That's not entirely true. Think of bug#18285, for example.

> In this case, all I
> have to offer is user experience, some requirements for what I
> consider to be a good solution, and some general guidelines for such a
> solution (which I only provided in response to repeated demands to do
> so).  If that is not useful, perhaps we should revise our instructions
> for bug reporting.

I'm sure it's useful in the general case. However, in certain situations 
a good design suggestion would be a better argument towards a change 
than only a feature request. You must be aware of that.

> IOW, I was reporting problems in an area where I know very little.  I
> don't think it's fair to demand that I provide, let alone code, a
> solution, as a prerequisite for acting on my report.

The amount of code that you'd have to look at is relatively small, in 
the current case. Just the xref-find-function docstring in xref.el, and 
the 70 lines at the end of etags.el.

> It is up to you to do whatever you want with this conjecture.  Some
> people pay a lot of money to get my conjectures in this and similar
> fields.  You get it for free.

Either way, this analysis doesn't look actionable enough on my side. Sorry.

> That's not an evidence of the design validity, not IME.  This feature
> is with us for too short time to be able to draw conclusions about its
> design.

Sure. We can't know for sure if it's valid. We could only find out that 
it's invalid, at some point.

> At least it "didn't work well enough" for me, which is a sign
> that it isn't perfect.  And you already made quite a few changes based
> on my experience.

Those changes didn't touch of the basics of the design, though. Just on 
implementations (which were initially not much more than proof-of-concept).

> I think it might be a good time to step back and
> review those changes, looking for some more fundamental issues that
> might benefit from some redesign.

Thanks, but this is, again, a very general kind of recommendation. It's 
not my first time writing code, too.

> Partly because CEDET is too heavyweight for most of my needs, and
> partly because I simply didn't have enough time to learn it.

Okay. But note that when you're asking for features that it already 
provides for some extent (semantic-symref supports renames; did you know 
that?), for us to create a better user experience we'll need members of 
the target audience to try their best to actually use it and report on 
the pain points.

> That's impossible.  I'm talking about projects whose line counts are
> in hundreds of thousands, sometimes millions.  You cannot read such
> project from top to bottom, when all you need to do is fix some bug or
> find the reason for some particular behavior:

If you can't read it whole, you continue to be in danger of malicious 
behavior tucked somewhere in the codebase. Would it really be better to 
leave it until production deployment, instead of allowing to happen on 
the developer's machine?

> IOW, your methodology would put the cart before the horse, in these
> use cases.

In any case, it's a well-known problem, and people have been known to 
live with that choice.

> I'm not necessarily saying I'm right in this case,
> but what if I am?  Shouldn't you at least consider that, instead of
> rejecting it flatly as "conjecture" and sticking to the original
> design as if it were a scripture?

I'm not rejecting. But like you said, one person can only do so much. 
When I don't see a clear path to action that would satisfy both the 
original requirements, as well as the ones you add, the estimated amount 
of work to get to the ideal is unbounded.

There are other features (as well as different projects) I'd like to 
work on.

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 15:41:22 +0300
> 
> On 05/02/2015 11:12 AM, Eli Zaretskii wrote:
> 
> > One person can only do so much, given her free time, and can only be
> > an expert in so many fields.  When you or Stefan report a problem with
> > the display engine, or in some other area where I know enough, I don't
> > ask for a design before I start working on it.
> 
> That's not entirely true. Think of bug#18285, for example.

Which part of what I said was not entirely true there?

> > In this case, all I
> > have to offer is user experience, some requirements for what I
> > consider to be a good solution, and some general guidelines for such a
> > solution (which I only provided in response to repeated demands to do
> > so).  If that is not useful, perhaps we should revise our instructions
> > for bug reporting.
> 
> I'm sure it's useful in the general case. However, in certain situations 
> a good design suggestion would be a better argument towards a change 
> than only a feature request. You must be aware of that.

If I had a good design suggestion, I'd certainly not withhold it.

> > IOW, I was reporting problems in an area where I know very little.  I
> > don't think it's fair to demand that I provide, let alone code, a
> > solution, as a prerequisite for acting on my report.
> 
> The amount of code that you'd have to look at is relatively small, in 
> the current case. Just the xref-find-function docstring in xref.el, and 
> the 70 lines at the end of etags.el.

xref.el uses facilities I never used, so for me it means I'd need to
study those as well.  Please believe me that when I said it's more
than I could invest, I already explored some of the code.

> > Partly because CEDET is too heavyweight for most of my needs, and
> > partly because I simply didn't have enough time to learn it.
> 
> Okay. But note that when you're asking for features that it already 
> provides for some extent (semantic-symref supports renames; did you know 
> that?), for us to create a better user experience we'll need members of 
> the target audience to try their best to actually use it and report on 
> the pain points.

I can be that user, at least in some cases.  Although the experience
of this bug report doesn't encourage me to get involved in more.

> > That's impossible.  I'm talking about projects whose line counts are
> > in hundreds of thousands, sometimes millions.  You cannot read such
> > project from top to bottom, when all you need to do is fix some bug or
> > find the reason for some particular behavior:
> 
> If you can't read it whole, you continue to be in danger of malicious 
> behavior tucked somewhere in the codebase. Would it really be better to 
> leave it until production deployment, instead of allowing to happen on 
> the developer's machine?

I'm talking about projects that are already deployed, sometimes for
years.

xref "find references" and grouping

[Dmitry Gutov]

On 05/02/2015 09:59 AM, Stefan Monnier wrote:

> I don't see much need for nesting.  I think the issue is mostly one of
> sorting in *xref* (or in the cycling, if we were to use a hypothetical
> other UI that provides functionality like the old etags.el).

Take a look at M-x semantic-symref output, and compare to what we have 
with M-x xref-find-references. Here's a screenshot of the former:

http://cedet.sourceforge.net/img-gen/symref.png

Note the grouping by the "current function", as well as by file name. It 
seems to me that it can be pretty valuable.

Here's how a similar interface looks in a popular proprietary IDE: 
https://www.jetbrains.com/img/webhelp/idea/findUsageField.png

Helmut, any suggestions? Does SLIME have anything related?

> So, let's focus on this.
> My suggestion would be either:
> - make xref-find-function return not just a list of locations, but also
>    an indication whether they're pre-sorted or not.
> - make xref-find-function return not a list of locations, but a list of
>    lists of locations (i.e. every location inside a sub-list is
>    considered to have the same "level/quality/likelihood", but the lists
>    are sorted by their "level/quality/likelihood").
> - let the backend provide its own sorting function to override the
>    current "group by file" sorting.

The question of sorting would need resolving indeed.

> Basically, I think the focus should be on making sure that we can
> re-implement the old behavior on top of the new API when using the
> xref/etags backend.

We also can't group xref-find-apropos matches by likelihood, I think. Or 
would you include file path on each line? That seems wasteful.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 03:57 PM, Eli Zaretskii wrote:

>> That's not entirely true. Think of bug#18285, for example.
>
> Which part of what I said was not entirely true there?

We've discussed the design, to some extent? And the tradeoffs?

At least that's how it looked for me.

> xref.el uses facilities I never used, so for me it means I'd need to
> study those as well.  Please believe me that when I said it's more
> than I could invest, I already explored some of the code.

I specifically mentioned only looking a xref-find-function in xref.el, 
not anything else. That's the whole point of the abstraction.

But I've no reason to disbelieve you anyway.

> I can be that user, at least in some cases.  Although the experience
> of this bug report doesn't encourage me to get involved in more.

You've got the "find references" implementation out of it, haven't you?

> I'm talking about projects that are already deployed, sometimes for
> years.

And yet you're worried what one of them could do to your machine when 
loaded into memory?

Re: UI inconveniences with M-.

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2015 16:31:13 +0300
> 
> On 05/02/2015 03:57 PM, Eli Zaretskii wrote:
> 
> >> That's not entirely true. Think of bug#18285, for example.
> >
> > Which part of what I said was not entirely true there?
> 
> We've discussed the design, to some extent? And the tradeoffs?

Yes, but I didn't demand that.

> > I can be that user, at least in some cases.  Although the experience
> > of this bug report doesn't encourage me to get involved in more.
> 
> You've got the "find references" implementation out of it, haven't you?

Yes, thanks.  I also got shouted at.

> > I'm talking about projects that are already deployed, sometimes for
> > years.
> 
> And yet you're worried what one of them could do to your machine when 
> loaded into memory?

Yes.

Re: xref "find references" and grouping

[Helmut Eller]

On Sat, May 02 2015, Dmitry Gutov wrote:

> Helmut, any suggestions?

Maybe the current group label could be generalized to a path in a tree
e.g. (<file> <class> <method>) where each element of such a tree path is
simply a string and the UI can display it either as a flat list or some
of kind tree widget.

> Does SLIME have anything related?

Not really.  For a time we grouped compiler error/warning/notes with a
kind of tree widget; the idea was to make it easier to skip over notes
and warnings and see error more quickly, but in practice much time was
spend on opening and closing those subtrees.  I don't know if a tree
widget is a win.  In Emacs such things always seem to require a lot of
extra screen space compared to the GUI widgets in other IDEs.

Helmut

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 04:44 PM, Eli Zaretskii wrote:

> Yes, but I didn't demand that.

There's no demand, I'd say. But the discussion has stopped making real 
progress without it.

The aforementioned bug is also at standstill, you might notice.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 10:59 AM, Helmut Eller wrote:

> Theoretically this can already be done by ignoring xref-location-group
> or by specializing xref-location-group for xref-etags-location so that it
> returns always the same group.

Or, if we try to be extra-informative, group by find-tag-tag-order 
functions (maybe replacing each name with some words in human language; 
or classify them into several groups, too). But we won't have any place 
to write file names to.

> Thinking about it: for etags it would make sense to group xrefs by name
> of the TAGS file because etags--xref-find-definitions doesn't sort xrefs
> "globally"; xrefs are reordered only within a TAGS file.

Indeed. But when combined with the above, it will be 2-3 levels of grouping.

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/02/2015 05:14 PM, Helmut Eller wrote:

> Maybe the current group label could be generalized to a path in a tree
> e.g. (<file> <class> <method>) where each element of such a tree path is
> simply a string and the UI can display it either as a flat list or some
> of kind tree widget.

In this format, we lose the ordering, though. Then we'll have sorting 
functions, which would need to be returned with the data, and we'll need 
to decide whether there's one function, or several (per level).

Or we could tackle this in OO fashion.

The different actions in xref-find-function will be required to return 
"renderable" elements (better name pending), in a backend-defined order, 
and there will be no subsequent sorting (basically, like now).

Each of those will implement a set of generic methods:

- Render itself in the xref buffer.
- Return the xref location element at point.
- Move point to the previous or next location element.

Later, some extra stuff, if we reach the "rename" feature, like:

- Return all locations inside.
- Mark or unmark all locations inside.

(Marked location elements would render themselves differently).

The downsides, as I see them:

- Less transparent data structures than a flat list.
- Some overhead on virtual dispatch (but hopefully it'll be negligible).
- Backends having a lot more say in how the buffer is displayed, and 
thus having more opportunities to break the rendering, use their own 
extra-special faces, and reinvent the interface (for better or worse).

The last point might also yield benefits: the generally agreed upon UI 
improvements could then be pushed into the default rendering logic.

> Not really.  For a time we grouped compiler error/warning/notes with a
> kind of tree widget; the idea was to make it easier to skip over notes
> and warnings and see error more quickly, but in practice much time was
> spend on opening and closing those subtrees.  I don't know if a tree
> widget is a win.  In Emacs such things always seem to require a lot of
> extra screen space compared to the GUI widgets in other IDEs.

Please note that folding (and extra indentation for sub-elements) is not 
a necessary element of what I'm describing. We can just use different 
faces for group headings, and otherwise render everything flat (if 
anyone disagrees, it's easy to make customizable).

Most of the time I dislike code folding, and the idea to conflate 
expanding of the matches with marking (for renaming) in semantic-symref 
strikes me as unfortunate. It still looks nice, though.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/02/2015 09:59 AM, Stefan Monnier wrote:

> Another thing that we should consider is how to let the user choose
> which backend to use.  Currently we only have xref-etags-mode, but
> I think it would make sense to make it possible for the user to
> dynamically choose among the potentially many more backends (e.g. we
> could add an xref/grep backend, and there will hopefulyl be more using
> things like id-utils, gtags, ...)

I still expect that managing those simply with minor modes will work 
well enough. While using id-utils or gtags instead of etags can be 
valuable, we'd probably stick to only one of them at a time.

Maybe choose between them automatically, like semantic-symref does (and 
allows us to do).

> and maybe also to use several backends
> at the same time (merging the results).

If we agree on the OO approach to grouping (or even the list-of-strings 
approach), merging them automatically would look pretty unsolvable to 
me. Concatenation would work, though (with the OO approach).

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 04/30/2015 08:46 PM, Helmut Eller wrote:

> next-error calls next-error-find-buffer.  Maybe we could record every
> buffer that was returned by next-error-find-buffer and include them in
> the candidate list.

That's not terrible, but then we have almost the same two non-ideal choices:

- Only record the buffers that have even been noticed as the value of 
next-error-last-buffer. This way we'll ignore e.g. Compilation buffers 
that have never had a chance to be used with `next-error'.

- Record all buffers that ever were returned from that function. This 
will also include normal (file-visiting) buffers where 
next-error-function is set locally (and usually only points to locations 
within that buffer).

Here's a different heuristic: return all buffers where 
next-error-function is set, but buffer-file-name is nil. Probably not 
ideal, but it'll clearly delineate between normal and special buffers.

I guess that will only leave inability, at certain times, to choose the 
current buffer's next-error-function over some Compilation buffer's one 
(http://debbugs.gnu.org/20489).

Re: conflicting uses of next-error-function

[Helmut Eller]

On Sun, May 03 2015, Dmitry Gutov wrote:

> On 04/30/2015 08:46 PM, Helmut Eller wrote:
>
>> next-error calls next-error-find-buffer.  Maybe we could record every
>> buffer that was returned by next-error-find-buffer and include them in
>> the candidate list.
>
> That's not terrible, but then we have almost the same two non-ideal choices:
>
> - Only record the buffers that have even been noticed as the value of
> next-error-last-buffer. This way we'll ignore e.g. Compilation buffers
> that have never had a chance to be used with `next-error'.
>
> - Record all buffers that ever were returned from that function. This
> will also include normal (file-visiting) buffers where
> next-error-function is set locally (and usually only points to
> locations within that buffer).

I was thinking of adding a new function like

 (next-error-note-buffer BUFFER &optional DONT-SELECT)

that essentially replaces next-error-last-buffer for those packages that
bother to use it AND to add some degree of backward compatibility by
recording the buffer returned by next-error-find-buffer.

Helmut

Re: xref "find references" and grouping

[Helmut Eller]

On Sat, May 02 2015, Dmitry Gutov wrote:

> On 05/02/2015 05:14 PM, Helmut Eller wrote:
>
>> Maybe the current group label could be generalized to a path in a tree
>> e.g. (<file> <class> <method>) where each element of such a tree path is
>> simply a string and the UI can display it either as a flat list or some
>> of kind tree widget.
>
> In this format, we lose the ordering, though. Then we'll have sorting
> functions, which would need to be returned with the data, and we'll
> need to decide whether there's one function, or several (per level).

Not sure what you mean.  Trees can be ordered and a list of tree-paths
would seem enough to describe an ordered tree.

[...]
> The different actions in xref-find-function will be required to return
> "renderable" elements (better name pending), in a backend-defined
> order, and there will be no subsequent sorting (basically, like now).

Sounds like there would not be much left to be shared between different
languages.  Maybe we should never have tried to generalize this anyway
and simply declare a convention that programming modes should bind
M-. to a find-definition-like command.  That certainly would avoid the
current dilemma of pleasing everybody.

[...]
> Later, some extra stuff, if we reach the "rename" feature, like:

A rename feature would probably benefit from an UI that's
separate/independent from find-definition.

Helmut

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/03/2015 09:54 AM, Helmut Eller wrote:

> I was thinking of adding a new function like
>
>   (next-error-note-buffer BUFFER &optional DONT-SELECT)
>
> that essentially replaces next-error-last-buffer for those packages that
> bother to use it AND to add some degree of backward compatibility by
> recording the buffer returned by next-error-find-buffer.

I'm not sure I understand your proposal. You seem to have ignored the 
description of the "non-ideal choice" number two.

Re: conflicting uses of next-error-function

[Helmut Eller]

On Sun, May 03 2015, Dmitry Gutov wrote:

> On 05/03/2015 09:54 AM, Helmut Eller wrote:
>
>> I was thinking of adding a new function like
>>
>>   (next-error-note-buffer BUFFER &optional DONT-SELECT)
>>
>> that essentially replaces next-error-last-buffer for those packages that
>> bother to use it AND to add some degree of backward compatibility by
>> recording the buffer returned by next-error-find-buffer.
>
> I'm not sure I understand your proposal. You seem to have ignored the
> description of the "non-ideal choice" number two.

Sorry, hadn't fully understood option two.  Yes, it option two probably
covers most cases that are relevant in practice.

Helmut

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/03/2015 04:25 PM, Helmut Eller wrote:

> Sorry, hadn't fully understood option two.  Yes, it option two probably
> covers most cases that are relevant in practice.

Here's the problem: with global-flycheck-mode on, all emacs-lisp-mode 
buffers have next-error-function set locally (to 
flycheck-next-error-function). And that function navigates between 
byte-compilation errors and warnings within the current buffer, which is 
very much in line with the next-error-function docstring (which says 
"find the next error in the current buffer").

Thus quite often next-error-find-buffer will return any emacs-lisp-mode 
buffer (for instance, when it's the only one visible in the current 
frame). If we remember all such buffers,

a) In a long-running Emacs session we'll remember a lot of them.

b) Setting next-error-last-buffer, while in one emacs-lisp-mode buffer, 
to another emacs-lisp-mode buffer doesn't make sense, because the latter 
only contains local error positions.

Thus, the resulting remembered list will be ill-suited for completion in 
the "change next-error-last-buffer" command.

Re: xref "find references" and grouping

[Stefan Monnier]

> languages.  Maybe we should never have tried to generalize this anyway
> and simply declare a convention that programming modes should bind
> M-. to a find-definition-like command.  That certainly would avoid the
> current dilemma of pleasing everybody.

Actually, the current tension doesn't come from xref.el but comes from
the fact that we now use find-func.el instead of etags.el by default in
Elisp mode buffers.  The use of xref.el should actually let use solve
this dilemma, rather than introduce it.


        Stefan

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/03/2015 10:47 AM, Helmut Eller wrote:

> Not sure what you mean.  Trees can be ordered and a list of tree-paths
> would seem enough to describe an ordered tree.

Ok, you're right. There's nothing impossible in a comparison function 
accepting lists of different lengths.

I can offer another, recently surfaced critique: if all groups are 
simply strings, we can't do content-specific things to them at display. 
For instance, can't shorten file paths.

Tree widgets can have specialized subclasses.

> Sounds like there would not be much left to be shared between different
> languages.

That's a very surprising conclusion. What prompted it? Like you said, 
we're not sorting now anyway. There's not much language-specific in the 
proposal.

A set of widgets (like generic group, file group, method group, and a 
few kinds of locations) will allow any language mode to return a 
structured set of matches, which can even be rendered in different ways.

> Maybe we should never have tried to generalize this anyway
> and simply declare a convention that programming modes should bind
> M-. to a find-definition-like command.  That certainly would avoid the
> current dilemma of pleasing everybody.

By allowing every mode to behave in its own way? That would certainly 
please every user.

> A rename feature would probably benefit from an UI that's
> separate/independent from find-definition.

It would be different, but not necessarily vastly different. And the set 
of returned xrefs we're getting from the "find references" command is 
already enough to base a simplistic rename on.

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Dmitry Gutov on Mon, 4 May 2015 05:09:17 +0300 wrote:

 > A set of widgets (like generic group, file group, method group, and a few kinds
 > of locations) will allow any language mode to return a structured set of
 > matches, which can even be rendered in different ways.

Ho about a simple tabular interface. Each back-end will return a list of
matches with a backend specific plist of metadata. For example:

  (foo . (:file "path/to/foo-file" :line 23 :priority 1 :type "method" :class "baz"))

Then arrange the xref in a simple flat table:

   File                  Priority  Type    Class  Target
   path/to/foo-file:23   1         method  bax    foo
   ...                   ...       ...     ...    ...

Then add a sorting command to sort on File, Priority Type etc. and
introduced a custom var for the default sorting.

There is a mode in emacs for tabular display, I forgot its name but I
think it already takes care of the sorting and other useful stuff.

Note that sorting with trees is not so straightforward. Depending on
sorting criteria you will get a very different trees in terms of inner
structure.

Trees also tend to waste a lot of space and are more demanding
cognitively because you need to figure out where one tree starts and
where it ends. As the "height" of each tree is different that could be a
rather hard task unless you use a clear delimiter between trees. But
that means using additional vertical space for delimiters.

One huge advantage of flat displays is that you can narrow candidates on
whatever component. So you can simply use selective display like occur,
helm-occur or ivy to narrow your buffer. You cannot do that with trees.

 Vitalie

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Stefan Monnier on Sat, 02 May 2015 02:59:28 -0400 wrote:

 > - make xref-find-function return not a list of locations, but a list of
 >   lists of locations (i.e. every location inside a sub-list is
 >   considered to have the same "level/quality/likelihood", but the lists
 >   are sorted by their "level/quality/likelihood").

That would allow for one hard-codded sorting. I think it's much better
to return a plist of metadata for each candidates. Then you can order on
multiple criteria.

 > - let the backend provide its own sorting function to override the
 >   current "group by file" sorting.

I think that's the task for the user to decide. The backend can compute
it's own :priority meta score and pass it back. If user's custom var
says sort on :priority then sort on priority.

Some backends might return more meta than others, so user's option
should be a list (:priority :type :file). The sorting algorithm should
got through the list and sort on first component which is defined. :file
must be defined for all backends.

 > I think it would make sense to make it possible for the user to
 > dynamically choose among the potentially many more backends (e.g. we
 > could add an xref/grep backend, and there will hopefulyl be more using
 > things like id-utils, gtags, ...) and maybe also to use several backends
 > at the same time (merging the results).  

Ho about the following UI. Instead of one set of backends you have
multiple levels of increasing sets of backends.

At each moment of time there is only one active "set" which you can
change with `xref-rotate-level`. The user then can customize levels of
backends:

  xref-backend-level-1 . (xref-default-mode-backend)
  xref-backend-level-2 . (xref-etags elisp-xref)
  xref-backend-level-3 . ....


A set corresponding to level N would automatically include all backends
from the lower levels. In the above example, if you set the level to 2
xref will merge default-backend, etags and elisp-xref.

Thus the UI is only 2 commands:

   xref-add-backend-in-level (ask for level and on prefix remove backend)
   xref-rotate-level (with numeric prefix, set level)

Users can then customize each level per mode or globally and thus have
consistent levels that they can rely upon independently of the context.

Vitalie

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Vitalie Spinu on Mon, 04 May 2015 15:10:35 +0200 wrote:
 > There is a mode in emacs for tabular display, I forgot its name but I
 > think it already takes care of the sorting and other useful stuff.

It's tabulated-list.el; same thing that package.el is based upon.

  Vitalie

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/04/2015 04:10 PM, Vitalie Spinu wrote:

> Note that sorting with trees is not so straightforward. Depending on
> sorting criteria you will get a very different trees in terms of inner
> structure.

You can still have a sorting command, and sort the output when it's a 
tree. The process would admittedly less obvious than with a tabulated 
list widget, though.

> Trees also tend to waste a lot of space and are more demanding

Again, we heavily disagree about what a waste of space is. In my mind, 
repeating the same value certainly qualifies.

> cognitively because you need to figure out where one tree starts and
> where it ends. As the "height" of each tree is different that could be a
> rather hard task unless you use a clear delimiter between trees. But
> that means using additional vertical space for delimiters.

There's no need for delimiters, I think. Just use different faces for 
group headers, and hope that tree branches of different lengths won't 
happen too often.

> One huge advantage of flat displays is that you can narrow candidates on
> whatever component. So you can simply use selective display like occur,
> helm-occur or ivy to narrow your buffer. You cannot do that with trees.

That's an advantage, yes. However, how are you even going to fit the 
xref-find-references output in a tabulated list? It contains full lines. 
If you put them in a column, the rest of the columns will surely overflow.

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/04/2015 04:41 PM, Vitalie Spinu wrote:

> At each moment of time there is only one active "set" which you can
> change with `xref-rotate-level`. The user then can customize levels of
> backends:
>
>    xref-backend-level-1 . (xref-default-mode-backend)
>    xref-backend-level-2 . (xref-etags elisp-xref)
>    xref-backend-level-3 . ....
>
>
> A set corresponding to level N would automatically include all backends
> from the lower levels. In the above example, if you set the level to 2
> xref will merge default-backend, etags and elisp-xref.

I'd rather not concern myself with this kind of complexity. However, if 
you're fine with merge logic simply concatenating the results (and it 
will be hard to do better), this shouldn't be hard to implement on top 
of the basic one-backend-at-a-time xref system, in a minor mode.

Re: conflicting uses of next-error-function

[Ted Zlatanov]

On Sun, 3 May 2015 17:12:42 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> Here's the problem: with global-flycheck-mode on, all emacs-lisp-mode
DG> buffers have next-error-function set locally (to
DG> flycheck-next-error-function). And that function navigates between
DG> byte-compilation errors and warnings within the current buffer, which
DG> is very much in line with the next-error-function docstring (which
DG> says "find the next error in the current buffer").

DG> Thus quite often next-error-find-buffer will return any
DG> emacs-lisp-mode buffer (for instance, when it's the only one visible
DG> in the current frame). If we remember all such buffers,

If we agree to implement a `next-error-priority', then perhaps
`next-error-find-buffer' should penalize buried buffers by subtracting
from their priority.  Thus buffers most recently buried will have the
lowest priority.

Ted

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 00:29:02 +0300 wrote:

 >> One huge advantage of flat displays is that you can narrow candidates on
 >> whatever component. So you can simply use selective display like occur,
 >> helm-occur or ivy to narrow your buffer. You cannot do that with trees.
 
 > That's an advantage, yes. However, how are you even going to fit the
 > xref-find-references output in a tabulated list?

Just make it the last column and abbreviate file paths and
indentation. xref-toggle-abbreviation is always an option,

Screens are big nowadays. Standard code is still 80 chars including
indentation.

 Vitalie

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 00:34:08 +0300 wrote:

 > I'd rather not concern myself with this kind of complexity. However,
 > if you're fine with merge logic simply concatenating the results (and
 > it will be hard to do better), this shouldn't be hard to implement on
 > top of the basic one-backend-at-a-time xref system, in a minor mode.

Sure thing. I can do the mode. How about the basic merging? Would it be
possible for you to put some basic stuff together?


  Vitalie

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/05/2015 03:13 AM, Vitalie Spinu wrote:

> Sure thing. I can do the mode. How about the basic merging? Would it be
> possible for you to put some basic stuff together?

What kind of stuff do you have in mind?

Re: xref "find references" and grouping

[Dmitry Gutov]

On 05/05/2015 03:02 AM, Vitalie Spinu wrote:
> xref-toggle-abbreviation is always an option,

And it would remove everything but the line content? So I'd toggle it on 
and off in order to know both?

> Screens are big nowadays. Standard code is still 80 chars including
> indentation.

Fully maximized Emacs frame is about 230 columns wide on my laptop. That 
allows to have two 113 column windows side-by-side. I'm pretty sure, 
with file paths and any other columns, the table width with go over 
those 113 columns.

Re: UI inconveniences with M-.

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 03:14:47 +0300 wrote:

 > On 05/05/2015 03:13 AM, Vitalie Spinu wrote:
 >> Sure thing. I can do the mode. How about the basic merging? Would it
 >> be possible for you to put some basic stuff together?

 > What kind of stuff do you have in mind?

I have in mind xref-find-functions list of functions instead of
xref-find-function symbol. But that's tricky because you have
xref-identifier-at-point-function and
xref-identifier-completion-table-function and potentially many more
coming soon. Each of these should also be lists to allow for multiple
backends.

I think a more condensed system would serve us much better here. Instead
of the above 3 config vars use one - `xref-backends` holding a list of
backend names. Like '(elisp etags). Then dynamically expand each of them
as in `elisp-xref-find`, `eslip-xref-identifier-at-point` and
`elisp-xref-completion-table`. If some of the these are not defined use
the default.

Then the minor mode as I have just proposed would be trivial to
implement. I don't see a straightforward implementation based on the
current xref.


  Vitalie





  

Re: xref "find references" and grouping

[Vitalie Spinu]

 >>> Dmitry Gutov on Tue, 5 May 2015 03:18:17 +0300 wrote:

 > On 05/05/2015 03:02 AM, Vitalie Spinu wrote:
 >> xref-toggle-abbreviation is always an option,

 > And it would remove everything but the line content? So I'd toggle it
 > on and off in order to know both?

That's interesting idea. Yes, I would like that (probably leaving just
the line numbers).

 >> Screens are big nowadays. Standard code is still 80 chars including
 >> indentation.

 > Fully maximized Emacs frame is about 230 columns wide on my
 > laptop. That allows to have two 113 column windows side-by-side. I'm
 > pretty sure, with file paths and any other columns, the table width
 > with go over those 113 columns.

240 on mine. I am rarely bothered with truncated lines with grep or
etags. It's commonly enough for me to see the beginning of the line. And
I am using M-g n and M-g p heavily. When I need full lines I simply
maximize the window. Toggling visibility would be much nicer
though. Especially if it could be done from outside of *xref* buffer
(M-g v?).


  Vitalie

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/05/2015 01:21 AM, Ted Zlatanov wrote:

> If we agree to implement a `next-error-priority', then perhaps
> `next-error-find-buffer' should penalize buried buffers by subtracting
> from their priority.  Thus buffers most recently buried will have the
> lowest priority.

Well yes, if we add a new variable buffers like *compile* set, we could 
detect them easily. And if we not only penalized but also removed 
buffers with priority 0, the list should be short enough.

But would we really have any priority values aside from 0 and 100? I 
think it makes sense for a buffer to only contain errors for itself, or 
only for other buffers.

Re: conflicting uses of next-error-function

[Ted Zlatanov]

On Tue, 5 May 2015 05:28:36 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> On 05/05/2015 01:21 AM, Ted Zlatanov wrote:
>> If we agree to implement a `next-error-priority', then perhaps
>> `next-error-find-buffer' should penalize buried buffers by subtracting
>> from their priority.  Thus buffers most recently buried will have the
>> lowest priority.

DG> Well yes, if we add a new variable buffers like *compile* set, we
DG> could detect them easily. And if we not only penalized but also
DG> removed buffers with priority 0, the list should be short enough.

Right.

DG> But would we really have any priority values aside from 0 and 100? I
DG> think it makes sense for a buffer to only contain errors for itself,
DG> or only for other buffers.

So far, we only know of those two cases, yes.  But we can adjust the
priority further by the buffer's stack position as I suggested above, or
by user-supplied criteria.  We could make it a floating-point value but
I'm not sure that would be a usability gain :)

Ted

Re: conflicting uses of next-error-function

[Dmitry Gutov]

On 05/05/2015 12:47 PM, Ted Zlatanov wrote:

> So far, we only know of those two cases, yes.  But we can adjust the
> priority further by the buffer's stack position as I suggested above, or
> by user-supplied criteria.  We could make it a floating-point value but
> I'm not sure that would be a usability gain :)

My instinct is to have a boolean var, and in case any new cases turn up, 
add a few new possible symbol values. For all we know, non-0, non-100 
situations might not be easily described with a number.

Re: conflicting uses of next-error-function

[Ted Zlatanov]

On Tue, 5 May 2015 17:05:43 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> On 05/05/2015 12:47 PM, Ted Zlatanov wrote:
>> So far, we only know of those two cases, yes.  But we can adjust the
>> priority further by the buffer's stack position as I suggested above, or
>> by user-supplied criteria.  We could make it a floating-point value but
>> I'm not sure that would be a usability gain :)

DG> My instinct is to have a boolean var, and in case any new cases turn
DG> up, add a few new possible symbol values. For all we know, non-0,
DG> non-100 situations might not be easily described with a number.

Sound good to me :)

Ted

Re: UI inconveniences with M-.

[Dmitry Gutov]

On 05/05/2015 04:36 AM, Vitalie Spinu wrote:
> Each of these should also be lists to allow for multiple
> backends.

List values don't make sense outside of the context of your minor mode, 
so I'd rather you do an external adapter converting a list to one backend.