bug#19466: 25.0.50; xref-find-def doesn't find C functions

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

Run "make TAGS" in the top-level directory, then:

 emacs -Q
 Click menu-bar->Edit->Go To->Set Tags File Name
 Navigate to src/TAGS and select it in the file selection dialog
 Click menu-bar->Edit->Go To->Find Definition
 Type "display_line RET" at the prompt
 => [No match]

But of course this function does exists and lives in xdisp.c.

The above does work if I invoke the same sequence from a buffer
visiting xdisp.c (or any other file from src/), but that sounds like
an inconvenience.  The old tags feature didn't require that.


In GNU Emacs 25.0.50.110 (i686-pc-mingw32)
 of 2014-12-29 on HOME-C4E4A596F7
Repository revision: ce1ebdf1ba8acc75e8f959f414652cdc87e76401
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --prefix=/d/usr --enable-checking=yes,glyphs 'CFLAGS=-O0
 -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Quit
Starting a new list of tags tables
Quit

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message dired format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils thingatpt vc-git diff-mode
easymenu easy-mmode etags xref eieio byte-opt bytecomp byte-compile
cl-extra cconv eieio-core cl-loaddefs cl-lib ring time-date tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 ls-lisp disp-table w32-win w32-vars tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)

Memory information:
((conses 8 91533 8774)
 (symbols 32 19742 0)
 (miscs 32 43 119)
 (strings 16 17923 4127)
 (string-bytes 1 497284)
 (vectors 8 12420)
 (vector-slots 4 417644 3202)
 (floats 8 71 100)
 (intervals 28 247 31)
 (buffers 516 12))

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 12/29/2014 09:27 PM, Eli Zaretskii wrote:
> Run "make TAGS" in the top-level directory, then:
>
>   emacs -Q
>   Click menu-bar->Edit->Go To->Set Tags File Name
>   Navigate to src/TAGS and select it in the file selection dialog
>   Click menu-bar->Edit->Go To->Find Definition
>   Type "display_line RET" at the prompt
>   => [No match]

Are you doing that in e.g. emacs-lisp-mode buffer? Naturally, it 
wouldn't work, because that major mode defines its own identifier 
completion table and find-definition function.

I understand what you're trying to do, but don't see a way to achieve 
that while keeping the uniform interface for the user in different major 
modes (which can use different navigation logic).

> The above does work if I invoke the same sequence from a buffer
> visiting xdisp.c (or any other file from src/),

Or buffer in any major mode that doesn't define its own navigation function.

 > but that sounds like
> an inconvenience.  The old tags feature didn't require that.

Suggestions welcome, but maybe you should just keep using `find-tag'. 
The generic navigation commands are more useful to have as the menu 
items, though.

Alternatively, you could reset `xref-find-function' and 
`xref-identifier-completion-table-function' to their default values in 
`emacs-lisp-mode-hook'. That could be a decent choice if your TAGS file 
includes the lisp files as well.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Tue, 30 Dec 2014 06:57:49 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 12/29/2014 09:27 PM, Eli Zaretskii wrote:
> 
>     Run "make TAGS" in the top-level directory, then:
> 
>       emacs -Q
>       Click menu-bar->Edit->Go To->Set Tags File Name
>       Navigate to src/TAGS and select it in the file selection dialog
>       Click menu-bar->Edit->Go To->Find Definition
>       Type "display_line RET" at the prompt
>       => [No match]
> 
> Are you doing that in e.g. emacs-lisp-mode buffer?

I did it in *scratch*.  But *scratch*'s major mode is not
emacs-lisp-mode, it's lisp-interaction-mode.  I think the difference
is significant for the purposes of this discussion.

> Naturally, it wouldn't work, because that major mode defines its own identifier completion table and find-definition function.
> 
> I understand what you're trying to do, but don't see a way to achieve that while keeping the uniform interface for the user in different major modes (which can use different navigation logic).

Isn't it possible to _prefer_ the symbols that are consistent with the
major mode, but not entirely discard the other possible candidates?

In a mixed-language project such as Emacs, these subtle conditions
that completely conceal symbols depending on the current mode, make
very little sense to me.  (And there are other mixed-language projects
out there, like Guile, GDB, etc.)  The Emacs TAGS tables traditionally
included tags from lisp/ files in src/TAGS, and for a very good
reason.

> Suggestions welcome, but maybe you should just keep using `find-tag'. The generic navigation commands are more useful to have as the menu items, though.

I assume that this new facility is an improvement, so I _want_ to use
it.  Especially since NEWS says:

  ** etags
  As a result of the above, these commands are now obsolete:
  `find-tag-other-window', `find-tag-other-frame', `find-tag-regexp',
  `tags-apropos' and `tags-loop-continue'.

Considering something obsolete means that a replacement is available
that is as good or better than the replaced feature.  I don't want to
go back to obsolete commands, unless I really have to, i.e. unless I
find the situation with xref unbearable.  I hope we are not there yet.

> Alternatively, you could reset `xref-find-function' and `xref-identifier-completion-table-function' to their default values in `emacs-lisp-mode-hook'. That could be a decent choice if your TAGS file includes the lisp files as well.

See above: out src/TAGS includes lisp/TAGS.  We do this for a long
time, and it's a tremendously useful thing when working on Emacs
development.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Stefan Monnier]

> Isn't it possible to _prefer_ the symbols that are consistent with the
> major mode, but not entirely discard the other possible candidates?

Indeed there should be a way to use the tags.el backend (or other
similar whole-project systems like GNU global, ...) instead of the
major mode's backend.


        Stefan

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 12/30/2014 08:05 PM, Stefan Monnier wrote:
>> Isn't it possible to _prefer_ the symbols that are consistent with the
>> major mode, but not entirely discard the other possible candidates?
>
> Indeed there should be a way to use the tags.el backend (or other
> similar whole-project systems like GNU global, ...) instead of the
> major mode's backend.

In general, a minor mode would do that (set project-specific 
xref-find-function and friends, etc), probably using a system like 
projectile to determine the current project kind.

We could do something specific for Emacs sources, but if using lisp/TAGS 
is preferred over find-func by the same developers who work on C 
sources, maybe emacs-lisp-mode shouldn't make any changes in xref-* 
variables for them. An boolean option would be the simplest solution.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Tue, 30 Dec 2014 20:25:25 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org
> 
> On 12/30/2014 08:05 PM, Stefan Monnier wrote:
> >> Isn't it possible to _prefer_ the symbols that are consistent with the
> >> major mode, but not entirely discard the other possible candidates?
> >
> > Indeed there should be a way to use the tags.el backend (or other
> > similar whole-project systems like GNU global, ...) instead of the
> > major mode's backend.
> 
> In general, a minor mode would do that (set project-specific 
> xref-find-function and friends, etc), probably using a system like 
> projectile to determine the current project kind.
> 
> We could do something specific for Emacs sources, but if using lisp/TAGS 
> is preferred over find-func by the same developers who work on C 
> sources, maybe emacs-lisp-mode shouldn't make any changes in xref-* 
> variables for them. An boolean option would be the simplest solution.

I probably don't understand exactly what you are suggesting, but can
we look at this from another angle: how can xref support looking up
symbols for projects that use more than one major mode?  E.g., could
the back-end be a list of functions, rather than just one?

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 12/30/2014 08:34 PM, Eli Zaretskii wrote:

> I probably don't understand exactly what you are suggesting, but can
> we look at this from another angle: how can xref support looking up
> symbols for projects that use more than one major mode?  E.g., could
> the back-end be a list of functions, rather than just one?

Like Stefan explained in the older discussion about this new feature 
(but in the context of `next-error-function'), `add-function' is 
probably the way to go for minor modes.

But that's not what I'm suggesting to do here.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 12/30/2014 05:31 PM, Eli Zaretskii wrote:

> I did it in *scratch*.  But *scratch*'s major mode is not
> emacs-lisp-mode, it's lisp-interaction-mode.  I think the difference
> is significant for the purposes of this discussion.

I wouldn't say it is (lisp-interaction-mode inherits from 
emacs-lisp-mode). And if we consider the scratch vs any .el file in the 
Emacs sources, it's less natural to use tags in scratch because it's 
conceptually farther from emacs/src/TAGS.

>> Naturally, it wouldn't work, because that major mode defines its own identifier completion table and find-definition function.
>>
>> I understand what you're trying to do, but don't see a way to achieve that while keeping the uniform interface for the user in different major modes (which can use different navigation logic).
>
> Isn't it possible to _prefer_ the symbols that are consistent with the
> major mode, but not entirely discard the other possible candidates?

Sure, but that has to be written in a sensible manner, too. Emacs core 
still has no notion of projects (or even moreso, project types), so it's 
harder to say "also use etags when in the Emacs sources".

Not all projects use etags, and this is almost never true in third-party 
Lisp projects, such as ELPA packages.

> In a mixed-language project such as Emacs, these subtle conditions
> that completely conceal symbols depending on the current mode, make
> very little sense to me.  (And there are other mixed-language projects
> out there, like Guile, GDB, etc.)  The Emacs TAGS tables traditionally
> included tags from lisp/ files in src/TAGS, and for a very good
> reason.

So if you're fine with tags, do you at all need the specialized 
navigation provided by emacs-lisp-mode? Like suggested, you could undo 
the changes made by emacs-lisp-mode to xref-* variables in 
emacs-lisp-mode-hook. A couple of `kill-local-variable' would suffice.

But personally, I'd never choose tags over find-func, and it's not a 
good default for the many users and third-party developers out there.

> Considering something obsolete means that a replacement is available
> that is as good or better than the replaced feature.  I don't want to
> go back to obsolete commands, unless I really have to, i.e. unless I
> find the situation with xref unbearable.  I hope we are not there yet.

Let's hope so.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Stefan Monnier]

> I probably don't understand exactly what you are suggesting, but can
> we look at this from another angle: how can xref support looking up
> symbols for projects that use more than one major mode?  E.g., could
> the back-end be a list of functions, rather than just one?

I think it's not just a matter of combining several major modes: you
need to know where the various files are and how they relate.  IOW there
needs to be some other logic on top of it and different systems will
require different such logic.


        Stefan

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 12/31/2014 12:44 AM, Stefan Monnier wrote:

> I think it's not just a matter of combining several major modes: you
> need to know where the various files are and how they relate.  IOW there
> needs to be some other logic on top of it and different systems will
> require different such logic.

Or, to put it another way: if someone is editing ELPA code in its own 
repo and presses `C-u M-.', I'm not sure it's reasonable to include all 
C functions and macros in the list of suggested identifiers. After all, 
they are writing Elisp, not C.

One way to deal with this would be to add a minor mode, one that people 
could enable on by-file or by-project basis (say, in Emacs's 
.dir-locals.el).

This xref-emacs-sources-mode would merge identifiers from both the Elisp 
and Etags backends, and try to merge the search results for different 
search actions as well.

It will return duplicate results, though. And filtering them out will be 
non-trivial.

For instance, if find-func and src/TAGS are merge, jumping to `car' 
would yield one xref-elisp-location, and one xref-file-location. How do 
you compare them, if one has fields with values "car" and "src/data.c", 
and the other "DEFUN ("car", Fcar," and "/full/p/t/emacs/src/data.c"?

Do we really want to include lisp/TAGS? It has both pros and cons.

Pros: we can jump to all functions, even non-autoloaded ones in all 
packages.

Cons: we do see all the functions, even ones we don't care about (their 
packages aren't loaded). Based on my positive experience with find-func, 
maybe 98% of the time the user won't really need them.

And duplicates, a lot more than with src/TAGS.

So someone should really consider if we ever need find-func and 
lisp/TAGS used together. Because if Eli only wants to use tag files when 
working on Emacs, the respective minor mode looks a lot simpler:

(defvar xref-etags-mode--save nil)

(define-minor-mode xref-etags-mode
   "Toggle using only the tag files for navigation."
   :lighter ""
   (if xref-etags-mode
       (progn
         (setq xref-etags-mode--save
               (cons xref-find-function
                     xref-identifier-completion-table-function))
         (kill-local-variable 'xref-find-function)
         (kill-local-variable 'xref-identifier-completion-table-function))
     (setq-local xref-find-function (car xref-etags-mode--save))
     (setq-local xref-identifier-completion-table-function
                 (cdr xref-etags-mode--save))))

Add it to find-file-hook, maybe predicated on buffer-file-name.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Stefan Monnier]

> So someone should really consider if we ever need find-func and lisp/TAGS
> used together. Because if Eli only wants to use tag files when working on
> Emacs, the respective minor mode looks a lot simpler:

Right, I think we should definitely start with just a way to use TAGS
*instead* of the major mode's own system.

We can consider later on adding some other system to merge TAGS info
with other info, but ... one step at a time, since we may end up not
needing the next step.


        Stefan

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  19466@debbugs.gnu.org
> Date: Sun, 04 Jan 2015 15:10:12 -0500
> 
> Right, I think we should definitely start with just a way to use TAGS
> *instead* of the major mode's own system.
> 
> We can consider later on adding some other system to merge TAGS info
> with other info, but ... one step at a time, since we may end up not
> needing the next step.

Btw, I think I've found another "issue" with xref-find-def: it uses
the TAGS table information without any tolerance.  So if you have a
TAGS file that is slightly outdated, you are put on the wrong line.
By contrast, etags.el had special code that would look around the
position specified by TAGS, see etags-goto-tag-location.

And one more thing: after invoking M-. and typing the function name,
then typing '.' to get the first candidate displayed, if I switch to
the window where that candidate is displayed, the window switches
buffers on me, so that the function I wanted to look at disappears.
What am I doing wrong this time?

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/04/2015 11:10 PM, Stefan Monnier wrote:

> Right, I think we should definitely start with just a way to use TAGS
> *instead* of the major mode's own system.

I've posted a tiny minor mode in the previous message that does this. 
Does it work well for everyone?

We probably don't even need a minor mode. A function might suffice:

(defun xref-reset-backend ()
   (kill-local-variable 'xref-find-function)
   (kill-local-variable 'xref-identifier-completion-table-function))

(defun elisp-maybe-reset-xref ()
   (when (and buffer-file-name
              (string-prefix-p source-directory buffer-file-name))
     (xref-reset-backend)))

(add-hook 'emacs-lisp-mode-hook 'elisp-maybe-reset-xref)

But please, someone else should make the choice: I rarely ever use tag 
files, and never with Emacs Lisp.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/04/2015 11:29 PM, Eli Zaretskii wrote:

> Btw, I think I've found another "issue" with xref-find-def: it uses
> the TAGS table information without any tolerance.  So if you have a
> TAGS file that is slightly outdated, you are put on the wrong line.
> By contrast, etags.el had special code that would look around the
> position specified by TAGS, see etags-goto-tag-location.

I see. Sounds like a good reason to add a yet-another xref-location 
subclass.

> And one more thing: after invoking M-. and typing the function name,
> then typing '.' to get the first candidate displayed, if I switch to
> the window where that candidate is displayed, the window switches
> buffers on me, so that the function I wanted to look at disappears.
> What am I doing wrong this time?

You didn't press RET after `.'? :)

Currently, pre-command-hook always restores window configuration to the 
one before the buffer with the definitions of the current line's xref 
was displayed.

I can see how this can be surprising for a new user, though. In that 
respect, a `quit-window' approach is better.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Mon, 05 Jan 2015 02:14:23 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org, Helmut Eller <eller.helmut@gmail.com>
> 
> > And one more thing: after invoking M-. and typing the function name,
> > then typing '.' to get the first candidate displayed, if I switch to
> > the window where that candidate is displayed, the window switches
> > buffers on me, so that the function I wanted to look at disappears.
> > What am I doing wrong this time?
> 
> You didn't press RET after `.'? :)

So there's no way of browsing the definition and keeping the list of
candidates available, except by scrolling the definition from another
window?  That's indeed surprising.

> Currently, pre-command-hook always restores window configuration to the 
> one before the buffer with the definitions of the current line's xref 
> was displayed.

Why is that insistence on restoring the window configuration useful?
Emacs users should be already used to the fact that Emacs pops buffers
in windows all the time, so IMO there's nothing terribly wrong in
leaving the window configuration changed.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Stefan Monnier]

>> Currently, pre-command-hook always restores window configuration to the 
>> one before the buffer with the definitions of the current line's xref 
>> was displayed.

> Why is that insistence on restoring the window configuration useful?
> Emacs users should be already used to the fact that Emacs pops buffers
> in windows all the time, so IMO there's nothing terribly wrong in
> leaving the window configuration changed.

Agreed.  In my book, any use of window-configuration restoration
is suspicious (too often used to hide an ugly "switch-to-buffer instead
of set-buffer", I guess).


        Stefan

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/05/2015 06:36 AM, Eli Zaretskii wrote:

> So there's no way of browsing the definition and keeping the list of
> candidates available, except by scrolling the definition from another
> window?  That's indeed surprising.

Right, maybe the quit-window approach would be less surprising overall.

The alternative implementation was pushed to scratch/xref, and it's 
waiting for Martin's comments (bug#19468).

> Why is that insistence on restoring the window configuration useful?
> Emacs users should be already used to the fact that Emacs pops buffers
> in windows all the time, so IMO there's nothing terribly wrong in
> leaving the window configuration changed.

Basically, I want the user to be able to easily return to the window 
configuration that was before xref-find-definitions was called. Or, if 
they picked a definition, make the process of selecting not leave its 
traces on the window configuration.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > The alternative implementation was pushed to scratch/xref, and it's waiting for Martin's comments (bug#19468).

Sorry, I didn't have time yet to give it a through testing.  Anyway,
Eli's concern

   "... if I switch to the window where that candidate is displayed, the
    window switches buffers on me ..."

is IMHO more serious than your

   "I want the user to be able to easily return to the window
    configuration that was before xref-find-definitions was called."

so I think you should push the `quit-window' based solution to trunk
(which, from the few testing I gave it, seems to handle this case well).


One thing that has annoyed me ever since is the buffers that pile up
while browsing tags.  I always dreamt of a good heuristic to get rid of
them.  Maybe you could try making provisions like

(1) this buffer is killable because it has been probably visited
     exclusively by and only accidentally during xrefing, and

(2) have the command that quits the *xref* buffer optionally kill the
     buffers marked in (1).

A heuristic for (1) could go as follows:

   The buffer was created during xrefing, the window was never selected
   while it showed that buffer and either its buffer was "immediately"
   replaced by another xrefed one or `xref--quit' was invoked.


You might also consider setting `other-window-scroll-buffer' to the
window used by `xref-show-location-at-point'.

martin, who'd prefer (user-error "No reference at point")

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Fri, 16 Jan 2015 06:37:46 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: monnier@iro.umontreal.ca, 19466@debbugs.gnu.org,
>  Helmut Eller <eller.helmut@gmail.com>,
>  martin rudalics <rudalics@gmx.at>
> 
> On 01/05/2015 06:36 AM, Eli Zaretskii wrote:
> 
> > So there's no way of browsing the definition and keeping the list of
> > candidates available, except by scrolling the definition from another
> > window?  That's indeed surprising.
> 
> Right, maybe the quit-window approach would be less surprising overall.
> 
> The alternative implementation was pushed to scratch/xref, and it's 
> waiting for Martin's comments (bug#19468).

Thanks.

> > Why is that insistence on restoring the window configuration useful?
> > Emacs users should be already used to the fact that Emacs pops buffers
> > in windows all the time, so IMO there's nothing terribly wrong in
> > leaving the window configuration changed.
> 
> Basically, I want the user to be able to easily return to the window 
> configuration that was before xref-find-definitions was called. Or, if 
> they picked a definition, make the process of selecting not leave its 
> traces on the window configuration.

I understand your desire.  I just think you are fighting a losing
battle, and in the process surprise the users with less than friendly
behavior.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Fri, 16 Jan 2015 08:47:55 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: monnier@iro.umontreal.ca, 19466@debbugs.gnu.org, 
>  Helmut Eller <eller.helmut@gmail.com>
> 
> One thing that has annoyed me ever since is the buffers that pile up
> while browsing tags.  I always dreamt of a good heuristic to get rid of
> them.

This happens to me all the time, but I don't recommend losing any
sleep over it.  It's very easy to delete the unwanted buffers (e.g.,
via "C-x C-b").  One does such cleanup anyway, when one finishes
working on some issue, right?

Emacs copes with many buffers just fine, so I wouldn't invest any
significant energy in getting rid of some of them using some
heuristics that will certainly misfire at some point.  Emacs should
only automatically kill buffers that it creates for its temporary
jobs, not buffers created by user commands.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > This happens to me all the time, but I don't recommend losing any
 > sleep over it.  It's very easy to delete the unwanted buffers (e.g.,
 > via "C-x C-b").  One does such cleanup anyway, when one finishes
 > working on some issue, right?

Till then they will show up, for example, via `previous-buffer'.

 > Emacs copes with many buffers just fine, so I wouldn't invest any
 > significant energy in getting rid of some of them using some
 > heuristics that will certainly misfire at some point.  Emacs should
 > only automatically kill buffers that it creates for its temporary
 > jobs, not buffers created by user commands.

It wouldn't kill them automatically but only optionally, via a special
command or a prefix.  Less heuristic than `clean-buffer-list' at least.

martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/16/2015 09:47 AM, martin rudalics wrote:

> so I think you should push the `quit-window' based solution to trunk
> (which, from the few testing I gave it, seems to handle this case well).

Pushed. I'd really like you to look at the implementation, though.

> One thing that has annoyed me ever since is the buffers that pile up
> while browsing tags.  I always dreamt of a good heuristic to get rid of

I thought of that problem too, but it seems less important than 
willy-nilly changing the window configuration, hence the currently 
discussed attempt to alleviate it.

> them.  Maybe you could try making provisions like
>
> (1) this buffer is killable because it has been probably visited
>      exclusively by and only accidentally during xrefing, and
>
> (2) have the command that quits the *xref* buffer optionally kill the
>      buffers marked in (1).

`xref--quit' could definitely be used for it.

> A heuristic for (1) could go as follows:
>
>    The buffer was created during xrefing, the window was never selected
>    while it showed that buffer and either its buffer was "immediately"
>    replaced by another xrefed one or `xref--quit' was invoked.

One question is, how will we know that if was never selected? Use a 
window-configuration-change-hook? Do we keep the newly added value there 
indefinitely, or when will `remove-hook' be called if the user never 
presses `q' in the xref buffer?

Overall, solving both problems would be easier if xref used a more 
restricting interface which would never allow to switch to the 
temporarily displayed buffers until the user made their choice (but 
sure, they could scroll the other window).

Maybe with `Electric-command-loop'.

> You might also consider setting `other-window-scroll-buffer' to the
> window used by `xref-show-location-at-point'.
>
> martin, who'd prefer (user-error "No reference at point")

Done.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > Pushed. I'd really like you to look at the implementation, though.

I didn't see any flaws in it.  In any case we'll find out pretty soon if
there are.

 >> (2) have the command that quits the *xref* buffer optionally kill the
 >>      buffers marked in (1).
 >
 > `xref--quit' could definitely be used for it.

That's what I thought.  Note in this context that some people might have
globally bound `quit-window' to some key other than 'q' and might want
to quit *xref* with that.  It's obviously up to them whether
`quit-window' does something additional in that context but it certainly
should do something "intuitively useful".  For example, with a prefix
argument it will kill the *xref* buffer and any hooks set up by xrefing
should get removed.

 > One question is, how will we know that if was never selected?

Via `buffer-list-update-hook'?  That is, if a buffer (1) was created by
xref but (2) never got "promoted" by `buffer-list-update-hook', then
such a buffer is eligible for being killed quietly.

 > Use a window-configuration-change-hook? Do we keep the newly added
 > value there indefinitely, or when will `remove-hook' be called if the
 > user never presses `q' in the xref buffer?

I'd say as soon as the *xref* buffer stops being displayed.  If a user
does that and a buffer gets killed by the way and the user later on
decides to redisplay that buffer via xrefing, she has to pay the price
of re-reading that buffer from file.  That's why this feature should be
optional.

 > Overall, solving both problems would be easier if xref used a more
 > restricting interface which would never allow to switch to the
 > temporarily displayed buffers until the user made their choice (but
 > sure, they could scroll the other window).
 >
 > Maybe with `Electric-command-loop'.

I profoundly dislike modal windows.  We should never restrict displaying
or perusing the *xref* buffer in any way.  Anything hardcoded here will
get you into troubles with users and inhibit developers to improve the
interface.

martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/19/2015 10:28 AM, martin rudalics wrote:

> I didn't see any flaws in it.  In any case we'll find out pretty soon if
> there are.

To me, xref--save-to-history looks like it's exploiting implementation 
details that could change in the future. Possibly because I haven't 
found good documentation for the quit-restore values.

> That's what I thought.  Note in this context that some people might have
> globally bound `quit-window' to some key other than 'q' and might want
> to quit *xref* with that.

You obviously mean [remap quit-window]. Forgot about that.

> It's obviously up to them whether
> `quit-window' does something additional in that context but it certainly
> should do something "intuitively useful".  For example, with a prefix
> argument it will kill the *xref* buffer and any hooks set up by xrefing
> should get removed.

I wouldn't say "prefix argument means kill" is inherently intuitive, but 
sure, a remapped command should mimic behavior of the original command 
as close as possible (I just wasn't aware of it).

>  > One question is, how will we know that if was never selected?
>
> Via `buffer-list-update-hook'?  That is, if a buffer (1) was created by
> xref but (2) never got "promoted" by `buffer-list-update-hook', then
> such a buffer is eligible for being killed quietly.

Sounds good. And as soon a it's run, it can change some local variable, 
and then the function will remove itself from the hook.

>  > Use a window-configuration-change-hook? Do we keep the newly added
>  > value there indefinitely, or when will `remove-hook' be called if the
>  > user never presses `q' in the xref buffer?
>
> I'd say as soon as the *xref* buffer stops being displayed.

Actually, with the above scheme there's no real need to call 
`remove-hook' from elsewhere. And if the xref buffer was buried 
sometime, then switched to again, and a buffer in question still wasn't 
selected in the meantime? Let it be killed, no great loss.

> If a user
> does that and a buffer gets killed by the way and the user later on
> decides to redisplay that buffer via xrefing, she has to pay the price
> of re-reading that buffer from file.  That's why this feature should be
> optional.

Does what?

I think it might be fine to always delete the "temporary" buffers if 
`xref--quit' was called with a prefix, as an initial implementation. 
After all, if the user doesn't mind having extra file buffers around, 
they'd also probably just bury xref.

> I profoundly dislike modal windows.  We should never restrict displaying
> or perusing the *xref* buffer in any way.  Anything hardcoded here will
> get you into troubles with users and inhibit developers to improve the
> interface.

I wouldn't necessarily agree (the keys and allowed commands could be 
customizable), but ok, let's not go that way.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Mon, 19 Jan 2015 09:28:35 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 19466@debbugs.gnu.org, Helmut Eller <eller.helmut@gmail.com>
> 
>  > Pushed. I'd really like you to look at the implementation, though.
> 
> I didn't see any flaws in it.  In any case we'll find out pretty soon if
> there are.

One issue I still see is that if TAGS is slightly outdated, point is
positioned not on the first line of a function/macro/struct, but on
the line recorded in TAGS.  I hope this will be fixed soon.

Another minor issue is with the help-echo in the xref buffer: it says
"mouse-2: display, mouse-1: navigate", which is confusing because it's
unclear what exactly each of these 2 means.  How about saying
explicitly "show in this window" and "show in another window"
(assuming this is what that does)?

Finally, I still think we need to allow searching symbols not only in
the current buffer's programming language, at least with the tags
back-end.  Without that, we cannot deprecate find-tag.

Thanks.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/19/2015 07:41 PM, Eli Zaretskii wrote:
>> Date: Mon, 19 Jan 2015 09:28:35 +0100
>> From: martin rudalics <rudalics@gmx.at>
>> CC: 19466@debbugs.gnu.org, Helmut Eller <eller.helmut@gmail.com>
>>
>>   > Pushed. I'd really like you to look at the implementation, though.
>>
>> I didn't see any flaws in it.  In any case we'll find out pretty soon if
>> there are.

The above refers to the window configuration-restoring logic.

> One issue I still see is that if TAGS is slightly outdated, point is
> positioned not on the first line of a function/macro/struct, but on
> the line recorded in TAGS.  I hope this will be fixed soon.

Okay, should work now. Thanks for the reminder.

> Another minor issue is with the help-echo in the xref buffer: it says
> "mouse-2: display, mouse-1: navigate", which is confusing because it's
> unclear what exactly each of these 2 means.  How about saying
> explicitly "show in this window" and "show in another window"
> (assuming this is what that does)?

Not exactly. Here "navigate" means bury the xref and other temporary 
buffers, and then display the reference in the current or other window, 
or frame it was originally intended to be displayed in (depending on 
which command was invoked: `xref-find-definitions', 
`xref-find-definitions-other-window' or 
`xref-find-definitions-other-frame').

> Finally, I still think we need to allow searching symbols not only in
> the current buffer's programming language, at least with the tags
> back-end.  Without that, we cannot deprecate find-tag.

Please refer to the following messages:

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19466#32
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19466#41

And for completeness, a third option: define a minor mode that would 
remap xref-find-* commands to their etags-only variants, which can be 
trivially implemented by let-binding the two relevant variables to their 
default values.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > To me, xref--save-to-history looks like it's exploiting implementation
 > details that could change in the future.

They could change, of course.  But since `xref--save-to-history'
exploits them, its designers should be able to participate in such
changes in the future.

 > Possibly because I haven't
 > found good documentation for the quit-restore values.

Hmm...  They are documented in the Window Parameters section.  What do
you miss?

 >> That's what I thought.  Note in this context that some people might have
 >> globally bound `quit-window' to some key other than 'q' and might want
 >> to quit *xref* with that.
 >
 > You obviously mean [remap quit-window].

No.  IMO it should be up to the user to remap `quit-window'.

 > I wouldn't say "prefix argument means kill" is inherently intuitive,

Backward compatibility.  That's also why `quit-window' has KILL as first
argument :-(

 > but sure, a remapped command should mimic behavior of the original
 > command as close as possible (I just wasn't aware of it).

What I meant was that people might be used to kill the *xref* buffer
along with closing its window or use `kill-buffer' directly on *xref*.
And that xref should be aware of that and act accordingly, for example,
by removing its hooks, _if any_.  Probably in `kill-buffer-hook'.

 >> If a user
 >> does that and a buffer gets killed by the way and the user later on
 >> decides to redisplay that buffer via xrefing, she has to pay the price
 >> of re-reading that buffer from file.  That's why this feature should be
 >> optional.
 >
 > Does what?

Remove *xref* from display, later on display *xref* again and try to
visit a buffer that was killed in between by xref.

 > I wouldn't necessarily agree (the keys and allowed commands could be
 > customizable), but ok, let's not go that way.

I don't want to impose my dislike on others.  But if you want to do
things modally you should give people means to optionally do things
manually.

IDE lovers, for example, might want to keep a small *xref* window
permanently open in some corner of the frame, with backward/forward
buttons for navigating their xref history.

And I'd probably plug in a timer-driven variant of `xref-next-line'
where moving to some particular line in the *xref* buffer window will
display the tag in another window, provided point remains sufficiently
long on that line.

martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/20/2015 10:01 AM, martin rudalics wrote:

> Hmm...  They are documented in the Window Parameters section.  What do
> you miss?

No, that's good enough, thanks. I was looking for that documentation in 
window.el.

>  >> Note in this context that some people might have
>  >> globally bound `quit-window' to some key other than 'q' and might want
>  >> to quit *xref* with that.
>  >
>  > You obviously mean [remap quit-window].
>
> No.  IMO it should be up to the user to remap `quit-window'.

If you're arguing to remove the `q' mapping in 
xref--xref-buffer-mode-map, that's a pretty circumstantial way to go 
about it. And I disagree.

The user is in control either way (they can modify the above map just as 
well), but the binding should be there by default because it's a part of 
the core functionality. `xref-goto-xref' also calls `xref--quit', and 
having one without the other by default would be weird.

> What I meant was that people might be used to kill the *xref* buffer
> along with closing its window or use `kill-buffer' directly on *xref*.
> And that xref should be aware of that and act accordingly, for example,
> by removing its hooks, _if any_.  Probably in `kill-buffer-hook'.

The hooks won't be a real problem. Currently, it sets up none. If we use 
buffer-list-update-hook, that it will fire at most once in each used 
buffer, and then remove itself. There's no harm leaving it in, and 
either way, it's a pretty unimportant detail.

> I don't want to impose my dislike on others.  But if you want to do
> things modally you should give people means to optionally do things
> manually.

Conceptually, doing things in a different way would correspond to a 
different value of xref-show-xrefs-function. Everyone's welcome to try 
creating a different interface.

> IDE lovers, for example, might want to keep a small *xref* window
> permanently open in some corner of the frame, with backward/forward
> buttons for navigating their xref history.

While I like the sound of this, I'm having hard time imagining how xrefs 
would work with back-forward buttons. Suppose the user performed a jump 
to a definition. What contents will xref buffer have after that? The same?

> And I'd probably plug in a timer-driven variant of `xref-next-line'
> where moving to some particular line in the *xref* buffer window will
> display the tag in another window, provided point remains sufficiently
> long on that line.

That's trivial to implement (I'm thinking post-command-hook with sit-for 
rather than a timer), but the hard part is creating, naming and 
documenting the yet-another user option, as far as I'm concerned. Feel 
free to try your hand at it.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > If you're arguing to remove the `q' mapping in
 > xref--xref-buffer-mode-map, that's a pretty circumstantial way to go
 > about it. And I disagree.

By no means.  `q' should do what it does now.

 > The user is in control either way (they can modify the above map just
 > as well), but the binding should be there by default because it's a
 > part of the core functionality. `xref-goto-xref' also calls
 > `xref--quit', and having one without the other by default would be
 > weird.

Sure.  All I mean is that if the user decides to bury the *xref* buffer
and later switches back to it, it should be functional as before.  And
if the user kills the buffer, there should not be any traces left.  Both
seem to work at the moment IIUC.  Here I only wanted to state that there
might be other means to stop/suspend xrefing than via `xref--quit'.

 > The hooks won't be a real problem. Currently, it sets up none. If we
 > use buffer-list-update-hook, that it will fire at most once in each
 > used buffer, and then remove itself. There's no harm leaving it in,
 > and either way, it's a pretty unimportant detail.

I think that the "remove itself" feature is good and if it can be done I
see no problems.

 > Conceptually, doing things in a different way would correspond to a
 > different value of xref-show-xrefs-function. Everyone's welcome to try
 > creating a different interface.

`xref--show-xref-buffer' is good as it is.  `xref-show-xrefs-function'
might be useful to emulate the old behavior where the buffer is never
shown but some keystroke is used to jump to the next tag.

 > While I like the sound of this, I'm having hard time imagining how
 > xrefs would work with back-forward buttons. Suppose the user performed
 > a jump to a definition. What contents will xref buffer have after
 > that? The same?

Yes.  But the buttons could get me to a buffer showing the result(s) of
the next and previous tag searches.  Basically what M-. followed by
previous-/next-history-element should do (and for some reason doesn't do
here yet).

 > That's trivial to implement (I'm thinking post-command-hook with
 > sit-for rather than a timer), but the hard part is creating, naming
 > and documenting the yet-another user option, as far as I'm
 > concerned. Feel free to try your hand at it.

I'll do that here as soon as I know where I want to place my *xref*
window within a frame.

In addition to what Eli reclaimed earlier: I need M-. to work in texi
buffers, the Emacs manuals, from *Help*, backtrace and customization
buffers.  How set things up for that?

Thanks, martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Tue, 20 Jan 2015 04:54:04 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> > One issue I still see is that if TAGS is slightly outdated, point is
> > positioned not on the first line of a function/macro/struct, but on
> > the line recorded in TAGS.  I hope this will be fixed soon.
> 
> Okay, should work now. Thanks for the reminder.

Thanks.

> > Another minor issue is with the help-echo in the xref buffer: it says
> > "mouse-2: display, mouse-1: navigate", which is confusing because it's
> > unclear what exactly each of these 2 means.  How about saying
> > explicitly "show in this window" and "show in another window"
> > (assuming this is what that does)?
> 
> Not exactly. Here "navigate" means bury the xref and other temporary 
> buffers, and then display the reference in the current or other window, 
> or frame it was originally intended to be displayed in (depending on 
> which command was invoked: `xref-find-definitions', 
> `xref-find-definitions-other-window' or 
> `xref-find-definitions-other-frame').

Then I'd suggest "show definition" instead of "navigate".  The latter
has no useful meaning in this context, and just confuses.

> > Finally, I still think we need to allow searching symbols not only in
> > the current buffer's programming language, at least with the tags
> > back-end.  Without that, we cannot deprecate find-tag.
> 
> Please refer to the following messages:
> 
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19466#32
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19466#41
> 
> And for completeness, a third option: define a minor mode that would 
> remap xref-find-* commands to their etags-only variants, which can be 
> trivially implemented by let-binding the two relevant variables to their 
> default values.

I don't really understand the difference between the various options,
so my suggestion would be to start with something that looks
promising, and then see if users like that.  The important thing is
implement something; just enumerating the alternatives is not enough.

Thanks.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/20/2015 07:24 PM, Eli Zaretskii wrote:

> Then I'd suggest "show definition" instead of "navigate".  The latter
> has no useful meaning in this context, and just confuses.

"go to definition" should be the clearest option, then.

...Except the xref interface is also supposed to be used for "apropos" 
and "find references". The latter, though not implemented for the two 
current backends, would be reasonably easy to do for certain 
environments like SLIME. "definition" won't be the right term then. Any 
suggestions?

> I don't really understand the difference between the various options,

I'm afraid you'll have to spend the effort to understand it (but feel 
free to ask questions). AFAICT, the audience for this feature is just a 
few people, and myself is not among them. So far, I don't have the 
proper requirements to work with.

Come on, in each case that's just a few lines of Lisp you need to look 
at, and maybe try.

> so my suggestion would be to start with something that looks
> promising, and then see if users like that.  The important thing is
> implement something; just enumerating the alternatives is not enough.

Messages 32 and 41 include functional implementations you can try. The 
patches that would go into Emacs won't be much different, we'd just have 
to decide on code organization.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/20/2015 04:51 PM, martin rudalics wrote:

> Sure.  All I mean is that if the user decides to bury the *xref* buffer
> and later switches back to it, it should be functional as before.  And
> if the user kills the buffer, there should not be any traces left.  Both
> seem to work at the moment IIUC.

Yeah, both should still work. I've pushed the buffer-killing 
implementation, please take a look.

One drawback comes to mind:

xref-goto-xref calls xref-quit without the KILL argument, so the 
temporary buffers are not cleared if you make a choice and press RET.

> previous-/next-history-element

Not sure what these are.

> In addition to what Eli reclaimed earlier: I need M-. to work in texi
> buffers, the Emacs manuals, from *Help*, backtrace and customization
> buffers.  How set things up for that?

Depends on what you want each of them to do.

debugger-mode should probably set both relevant vars to the same values 
as emacs-lisp-mode. help-mode and Custom-mode - maybe too, although they 
might use some custom logic.

In Info-mode, xref-find-function could use the index and the search 
functionality.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > Yeah, both should still work. I've pushed the buffer-killing implementation, please take a look.

The idea to do it via the KILL argument is good.  I think some
doc-strings are bad.  For example, instead of

(defvar-local xref--selected nil
   "t if the current buffer has ever been selected.
Used for temporary buffers.")

I'd say something like

(defvar-local xref--current nil
   "Non-nil if this buffer was current once while finding xrefs.")

And for `xref-quit' I'd describe the standard behavior first and the
KILL behavior afterwards.  Also I'm not sure it it's just cosmetics but
shouldn't the

     (pcase-dolist (`(,buf . ,win) history)
       (when (and (window-live-p win)
                  (eq buf (window-buffer win)))
         (quit-window nil win)))

precede the

     (when kill
       (let ((xref--inhibit-mark-selected t)
             kill-buffer-query-functions)
         (dolist (buf xref--temporary-buffers)
           (unless (buffer-local-value 'xref--selected buf)
             (kill-buffer buf)))
         (setq xref--temporary-buffers nil)))

part?  I can't test it currently because I always get

Debugger entered--Lisp error: (args-out-of-range "" 0)
   help-function-arglist(#[257 "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"] preserve-names)
   eieio--defmethod(xref-location-group nil xref-bogus-location #[257 "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"])
   byte-code("\300\301\302\301\303\"\"\210\304\301\303\305\306$\210\300\307\302\307\303\"\"\210\304\307\303\305\310$\207" [eieio--defalias xref-location-marker eieio--defgeneric-init-form nil eieio--defmethod xref-bogus-location #[257 "\300\301\302\x03\303\"\"\207" [user-error "%s" eieio-oref :message] 6 "\n\n(fn L)"] xref-location-group #[257 "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"]] 5)
   autoload-do-load((autoload "xref" 1640080 t nil) xref-find-definitions)
   command-execute(xref-find-definitions)

even after a bootstrap.

 > One drawback comes to mind:
 >
 > xref-goto-xref calls xref-quit without the KILL argument, so the temporary buffers are not cleared if you make a choice and press RET.

You can redisplay the *xref* buffer and provide the KILL there.

 >> previous-/next-history-element
 >
 > Not sure what these are.

Something like typing M-. doing something else and typing M-. again.  At
this time up/down should get you the next/previous history elements of
your xref searches.

 > Depends on what you want each of them to do.
 >
 > debugger-mode should probably set both relevant vars to the same values as emacs-lisp-mode. help-mode and Custom-mode - maybe too, although they might use some custom logic.
 >
 > In Info-mode, xref-find-function could use the index and the search functionality.

Will there be a canonical way to add these?

martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/21/2015 12:49 PM, martin rudalics wrote:

> The idea to do it via the KILL argument is good.  I think some
> doc-strings are bad.  For example, instead of
>
> (defvar-local xref--selected nil
>    "t if the current buffer has ever been selected.
> Used for temporary buffers.")
>
> I'd say something like
>
> (defvar-local xref--current nil
>    "Non-nil if this buffer was current once while finding xrefs.")

"current" would be good, but "while finding xrefs" is iffy: we actually 
try to ignore instances of buffers being current while we display them 
(see xref--inhibit-mark-selected).

> And for `xref-quit' I'd describe the standard behavior first and the
> KILL behavior afterwards.

Would you like to suggest a specific wording?

> Also I'm not sure it it's just cosmetics but
> shouldn't the
>
>      (pcase-dolist (`(,buf . ,win) history)
>        (when (and (window-live-p win)
>                   (eq buf (window-buffer win)))
>          (quit-window nil win)))
>
> precede the
>
>      (when kill
>        (let ((xref--inhibit-mark-selected t)
>              kill-buffer-query-functions)
>          (dolist (buf xref--temporary-buffers)
>            (unless (buffer-local-value 'xref--selected buf)
>              (kill-buffer buf)))
>          (setq xref--temporary-buffers nil)))
>
> part?

Probably, but there's likely not much difference at the moment. xref 
doesn't pop any new windows, so we shouldn't miss on deleting those when 
quitting temporary buffers.

> I can't test it currently because I always get
>
> Debugger entered--Lisp error: (args-out-of-range "" 0)
>    help-function-arglist(#[257 "\300\207" ["(No location)"] 2 "(No
> location)\n\n(fn ##)"] preserve-names)
>    eieio--defmethod(xref-location-group nil xref-bogus-location #[257
> "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"])

I'm sure Someone(tm) will fix that right away. :)

>  > xref-goto-xref calls xref-quit without the KILL argument, so the
> temporary buffers are not cleared if you make a choice and press RET.
>
> You can redisplay the *xref* buffer and provide the KILL there.

You mean, for the user to switch to the buried xref buffer, then press 
C-u q? That's quite a few keypresses.

>  >> previous-/next-history-element
>  >
>  > Not sure what these are.
>
> Something like typing M-. doing something else and typing M-. again.  At
> this time up/down should get you the next/previous history elements of
> your xref searches.

You mean in the current interface? I'm confused.

>  > debugger-mode should probably set both relevant vars to the same
> values as emacs-lisp-mode. help-mode and Custom-mode - maybe too,
> although they might use some custom logic.
>  >
>  > In Info-mode, xref-find-function could use the index and the search
> functionality.
>
> Will there be a canonical way to add these?

I meant that xref-identifier-completion-table-function would return 
identifiers from the index, and xref-find-function would delegate to the 
search functionality.

With xref, we already sorta have both as features (C-u M-x 
xref-find-definitions and M-x xref-find-apropos).

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Tue, 20 Jan 2015 22:49:39 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 01/20/2015 07:24 PM, Eli Zaretskii wrote:
> 
> > Then I'd suggest "show definition" instead of "navigate".  The latter
> > has no useful meaning in this context, and just confuses.
> 
> "go to definition" should be the clearest option, then.

Works for me.

> ...Except the xref interface is also supposed to be used for "apropos" 
> and "find references". The latter, though not implemented for the two 
> current backends, would be reasonably easy to do for certain 
> environments like SLIME. "definition" won't be the right term then. Any 
> suggestions?

Can the package that uses xref modify that string?  If so, we don't
need a single-fits-all phrase.

If we do need a single phrase, then how about "go to
definition/reference"?

> > I don't really understand the difference between the various options,
> 
> I'm afraid you'll have to spend the effort to understand it (but feel 
> free to ask questions). AFAICT, the audience for this feature is just a 
> few people, and myself is not among them. So far, I don't have the 
> proper requirements to work with.

Perhaps you could ask what issues need to be resolved for you to
understand the requirements.

> Come on, in each case that's just a few lines of Lisp you need to look 
> at, and maybe try.

If you can show me the complete Lisp, I can try that and return
feedback.

> Messages 32 and 41 include functional implementations you can try. The 
> patches that would go into Emacs won't be much different, we'd just have 
> to decide on code organization.

The recipe in #32 is clearly incomplete.  Not sure about 41.

Once again, if you can show a complete Lisp to try, I will.

Thanks.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > "current" would be good, but "while finding xrefs" is iffy: we actually try to ignore instances of buffers being current while we display them (see xref--inhibit-mark-selected).

In any case please stick to the "`current' is a predicate for buffers
and `selected' is a predicate for windows and frames" convention.

 >> And for `xref-quit' I'd describe the standard behavior first and the
 >> KILL behavior afterwards.
 >
 > Would you like to suggest a specific wording?

I'll try to as soon as I know what it does.  Usually the "If optional
argument KILL is non-nil ..." idiom should appear after describing the
normal behavior.

 >> Also I'm not sure it it's just cosmetics but
 >> shouldn't the
 >>
 >>      (pcase-dolist (`(,buf . ,win) history)
 >>        (when (and (window-live-p win)
 >>                   (eq buf (window-buffer win)))
 >>          (quit-window nil win)))
 >>
 >> precede the
 >>
 >>      (when kill
 >>        (let ((xref--inhibit-mark-selected t)
 >>              kill-buffer-query-functions)
 >>          (dolist (buf xref--temporary-buffers)
 >>            (unless (buffer-local-value 'xref--selected buf)
 >>              (kill-buffer buf)))
 >>          (setq xref--temporary-buffers nil)))
 >>
 >> part?
 >
 > Probably, but there's likely not much difference at the moment. xref
 > doesn't pop any new windows,

... well, it did so here all the time ...

 > so we shouldn't miss on deleting those when quitting temporary buffers.

Are you sure?  Suppose `xref-show-location-at-point' reused a window
showing a buffer A for showing a buffer B instead.  Now `xref-quit'
would kill B which should again show A instead.  But here you rely on
`kill-buffer' (that is `replace-buffer-in-windows') DTRT.

 >> I can't test it currently because I always get
 >>
 >> Debugger entered--Lisp error: (args-out-of-range "" 0)
 >>    help-function-arglist(#[257 "\300\207" ["(No location)"] 2 "(No
 >> location)\n\n(fn ##)"] preserve-names)
 >>    eieio--defmethod(xref-location-group nil xref-bogus-location #[257
 >> "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"])
 >
 > I'm sure Someone(tm) will fix that right away. :)

For some value of "right away" :)

 >>  > xref-goto-xref calls xref-quit without the KILL argument, so the
 >> temporary buffers are not cleared if you make a choice and press RET.
 >>
 >> You can redisplay the *xref* buffer and provide the KILL there.
 >
 > You mean, for the user to switch to the buried xref buffer, then press C-u q? That's quite a few keypresses.

Then you probably want the KILL argument for `xref-goto-xref' too.

 >>  >> previous-/next-history-element
 >>  >
 >>  > Not sure what these are.
 >>
 >> Something like typing M-. doing something else and typing M-. again.  At
 >> this time up/down should get you the next/previous history elements of
 >> your xref searches.
 >
 > You mean in the current interface? I'm confused.

When xref prompts me with

Find definitions of:

I'd expect to be able to browse my history of definitions I tried or
succeeded to find.

 > With xref, we already sorta have both as features (C-u M-x xref-find-definitions and M-x xref-find-apropos).

OK.  They all work well in my gdb buffer.  So it shouldn't be too hard.

martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 >  >> Debugger entered--Lisp error: (args-out-of-range "" 0)
 >  >>    help-function-arglist(#[257 "\300\207" ["(No location)"] 2 "(No
 >  >> location)\n\n(fn ##)"] preserve-names)
 >  >>    eieio--defmethod(xref-location-group nil xref-bogus-location #[257
 >  >> "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"])
 >  >
 >  > I'm sure Someone(tm) will fix that right away. :)
 >
 > For some value of "right away" :)

Meanwhile I additionally get:

   ELC      progmodes/xref.elc
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
`defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
`defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.

In end of data:
progmodes/xref.el:625:1:Warning: the function `apropos-parse-pattern' is not
     known to be defined.

martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/21/2015 06:54 PM, martin rudalics wrote:

> In any case please stick to the "`current' is a predicate for buffers
> and `selected' is a predicate for windows and frames" convention.

How about `visited'? The other precise alternative for that variable 
named is `was-current'.

And considering that our main caller of that hook would be 
`select-window', maybe the current name isn't too far from the mark?

> Are you sure?  Suppose `xref-show-location-at-point' reused a window
> showing a buffer A for showing a buffer B instead.  Now `xref-quit'
> would kill B which should again show A instead.  But here you rely on
> `kill-buffer' (that is `replace-buffer-in-windows') DTRT.

Right, I do rely on kill-buffer doing what I'd expect. Anyway, I'm fine 
with swapping the order, will do so soon.

>  > You mean, for the user to switch to the buried xref buffer, then
> press C-u q? That's quite a few keypresses.
>
> Then you probably want the KILL argument for `xref-goto-xref' too.

Maybe. Wouldn't an option server this better?

We also allow jumping to a mouse click, and pressing C-u in that case 
becomes more awkward.

> When xref prompts me with
>
> Find definitions of:
>
> I'd expect to be able to browse my history of definitions I tried or
> succeeded to find.

Ok, thanks. I'll look into input history.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Stefan Monnier]

>> >> Debugger entered--Lisp error: (args-out-of-range "" 0)
>> >>    help-function-arglist(#[257 "\300\207" ["(No location)"] 2 "(No
>> >> location)\n\n(fn ##)"] preserve-names)
>> >>    eieio--defmethod(xref-location-group nil xref-bogus-location #[257
>> >> "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"])
>> > I'm sure Someone(tm) will fix that right away. :)

Sorry, I missed this.  When/how do you get this error?

> Meanwhile I additionally get:
>   ELC      progmodes/xref.elc
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.
> `defmethod' is an obsolete macro (as of 25.1); use `cl-defmethod' instead.
> `defgeneric' is an obsolete macro (as of 25.1); use `cl-defgeneric' instead.

There are two problems here:
1- The warnings fail to include the "xref.el:<lineno>: " prefix.
   Haven't figured out why yet.  If someone wants to help fix this, he'd
   be very welcome.  The problem seems to happen only when
   macroexp--warn-and-return is used on a toplevel form (aka
   "file-form").  The warnings themselves are otherwise correct (and
   "harmless").
2- xref.el (just like pretty much all existing code using EIEIO) uses
   obsolete macros, indeed.  This is easy to fix.


        Stefan

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/21/2015 06:54 PM, martin rudalics wrote:

> In any case please stick to the "`current' is a predicate for buffers
> and `selected' is a predicate for windows and frames" convention.

Anyway, xref--current seems good enough.

> I'll try to as soon as I know what it does.  Usually the "If optional
> argument KILL is non-nil ..." idiom should appear after describing the
> normal behavior.

I've updated it in that direction.

> When xref prompts me with
>
> Find definitions of:
>
> I'd expect to be able to browse my history of definitions I tried or
> succeeded to find.

This should work now.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/21/2015 10:02 PM, Stefan Monnier wrote:

> Sorry, I missed this.  When/how do you get this error?

I happened on evaluating xref.el after you do 'make' in Emacs sources.

> There are two problems here:
> 1- The warnings fail to include the "xref.el:<lineno>: " prefix.
>     Haven't figured out why yet.  If someone wants to help fix this, he'd
>     be very welcome.  The problem seems to happen only when
>     macroexp--warn-and-return is used on a toplevel form (aka
>     "file-form").  The warnings themselves are otherwise correct (and
>     "harmless").
> 2- xref.el (just like pretty much all existing code using EIEIO) uses
>     obsolete macros, indeed.  This is easy to fix.

Guess which one I've just fixed. :)

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/21/2015 06:25 PM, Eli Zaretskii wrote:

> Can the package that uses xref modify that string?  If so, we don't
> need a single-fits-all phrase.

It can't, right now. But it would have to be modified by the used 
command (find-definitions/references/apropos), not the package.

> If we do need a single phrase, then how about "go to
> definition/reference"?

I've settled on "follow reference" for now.

> Perhaps you could ask what issues need to be resolved for you to
> understand the requirements.

Someone would have to gather them, and participate in the discussion 
about the tradeoffs.

AFAICS you've pretty much stopped formulating the requirements at "there 
should be a way to use the tags.el backend instead of the major mode's 
backend". That's not good enough for me.

> If you can show me the complete Lisp, I can try that and return
> feedback.

That would be good, but unless you declare one of them good enough, 
participation in the technical discussion will also be needed.

>> Messages 32 and 41 include functional implementations you can try. The
>> patches that would go into Emacs won't be much different, we'd just have
>> to decide on code organization.
>
> The recipe in #32 is clearly incomplete.  Not sure about 41.

It's only incomplete because I expected you'd be able to turn on the 
minor mode defined there yourself, in any Emacs Lisp buffers you like.

Here's the "missing piece" for #32:

(defun elisp-maybe-turn-on-xref-etags-mode ()
   (when (and buffer-file-name
              (string-prefix-p source-directory buffer-file-name))
     (xref-etags-mode)))

(add-hook 'emacs-lisp-mode-hook 'elisp-maybe-turn-on-xref-etags-mode)

Which part of it was hard?

> Once again, if you can show a complete Lisp to try, I will.

To my knowledge, #41 was/is quite complete.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Thu, 22 Jan 2015 04:43:47 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 01/21/2015 06:25 PM, Eli Zaretskii wrote:
> 
> > Can the package that uses xref modify that string?  If so, we don't
> > need a single-fits-all phrase.
> 
> It can't, right now. But it would have to be modified by the used 
> command (find-definitions/references/apropos), not the package.
> 
> > If we do need a single phrase, then how about "go to
> > definition/reference"?
> 
> I've settled on "follow reference" for now.

Probably good enough, thanks.

> AFAICS you've pretty much stopped formulating the requirements at "there 
> should be a way to use the tags.el backend instead of the major mode's 
> backend".

Not really, I think that was Stefan's proposal.

What I need is a way to find definitions of both C and Lisp symbols
(functions, macros, struct's, etc.) irrespective of the current
buffer's major mode.  If xref can do that, it's fine with me.

> > If you can show me the complete Lisp, I can try that and return
> > feedback.
> 
> That would be good, but unless you declare one of them good enough, 
> participation in the technical discussion will also be needed.
> 
> >> Messages 32 and 41 include functional implementations you can try. The
> >> patches that would go into Emacs won't be much different, we'd just have
> >> to decide on code organization.
> >
> > The recipe in #32 is clearly incomplete.  Not sure about 41.
> 
> It's only incomplete because I expected you'd be able to turn on the 
> minor mode defined there yourself, in any Emacs Lisp buffers you like.
> 
> Here's the "missing piece" for #32:
> 
> (defun elisp-maybe-turn-on-xref-etags-mode ()
>    (when (and buffer-file-name
>               (string-prefix-p source-directory buffer-file-name))
>      (xref-etags-mode)))
> 
> (add-hook 'emacs-lisp-mode-hook 'elisp-maybe-turn-on-xref-etags-mode)
> 
> Which part of it was hard?
> 
> > Once again, if you can show a complete Lisp to try, I will.
> 
> To my knowledge, #41 was/is quite complete.

I will try those when I have time, thanks.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 >>>>> Debugger entered--Lisp error: (args-out-of-range "" 0)
 >>>>>     help-function-arglist(#[257 "\300\207" ["(No location)"] 2 "(No
 >>>>> location)\n\n(fn ##)"] preserve-names)
 >>>>>     eieio--defmethod(xref-location-group nil xref-bogus-location #[257
 >>>>> "\300\207" ["(No location)"] 2 "(No location)\n\n(fn ##)"])
 >>>> I'm sure Someone(tm) will fix that right away. :)
 >
 > Sorry, I missed this.  When/how do you get this error?

Never mind.  You fixed it already ;-)

 > There are two problems here:
 > 1- The warnings fail to include the "xref.el:<lineno>: " prefix.
 >     Haven't figured out why yet.  If someone wants to help fix this, he'd
 >     be very welcome.  The problem seems to happen only when
 >     macroexp--warn-and-return is used on a toplevel form (aka
 >     "file-form").  The warnings themselves are otherwise correct (and
 >     "harmless").
 > 2- xref.el (just like pretty much all existing code using EIEIO) uses
 >     obsolete macros, indeed.  This is easy to fix.

These have been fixed as well.

Thanks, martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[martin rudalics]

 > Anyway, xref--current seems good enough.

It's OK now.

 > I've updated it in that direction.

OK as well.

 >> When xref prompts me with
 >>
 >> Find definitions of:
 >>
 >> I'd expect to be able to browse my history of definitions I tried or
 >> succeeded to find.
 >
 > This should work now.

It does.

Thanks, martin

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/22/2015 08:02 PM, Eli Zaretskii wrote:

> What I need is a way to find definitions of both C and Lisp symbols
> (functions, macros, struct's, etc.) irrespective of the current
> buffer's major mode.  If xref can do that, it's fine with me.

Would it also be irrespective of the current file, or its project? Would 
it not depend on major mode at all, so it would also be true in 
help-mode and similar buffers?

If you want it in all major modes, you can use find-file-hook instead of 
emacs-lisp-mode-hook. If in all files everywhere, then you can also drop 
the buffer-file-name check, ending up with

(add-hook 'find-file-hook #'xref-etags-mode t)

That doesn't help with non-file buffers, though. But you can use a 
separate major mode hook for each.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Thu, 22 Jan 2015 23:02:22 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 01/22/2015 08:02 PM, Eli Zaretskii wrote:
> 
> > What I need is a way to find definitions of both C and Lisp symbols
> > (functions, macros, struct's, etc.) irrespective of the current
> > buffer's major mode.  If xref can do that, it's fine with me.
> 
> Would it also be irrespective of the current file, or its project?

Most, if not all, projects I work on aren't "projects" in your sense
of the word, I think -- they lack any "project" files except the
sources and Makefile's.  With etags, I switch "projects" by invoking
visit-tags-table and typing a name of a different TAGS file (I'm then
given a choice of whether to keep the previous TAGS table or discard
it, which is important -- see below).

If xref can reliably deduce that I switched projects and automatically
update its database, that's fine with me, and would probably
constitute what you mean by "dependence on the current file or
project".  But I doubt that this can be done reliably enough to
satisfy users like me, when a project doesn't have a definitive
description of what is or isn't part of it.

Take, for example, the use case where I'm testing a program and found
a bug.  I then need to be able to quickly find and examine the
definition of symbols that might be involved in the bug, look at their
code, perhaps make some changes -- this all will be served well by
using the database (such as TAGS) of that single program.  But suppose
I now come to the conclusion that the bug is not in the program per
se, but involves one of the external libraries it uses.  Now I need to
go through the sources of that library (whose sources, by sheer luck
or maybe something else, I already have available on my system).  How
would xref or etags know whether I switched to that library as part of
my previous work (and therefore still need access to the previous
project's symbols), or because I'm now working on an entirely
different project?  What if investigating the bug needs to
intermittently look at the sources of the program and the library
(e.g., because the bug happens due to some incompatibility between
them)?

So I think there will always be a need for asking the user about this,
and in addition there are projects without "project" infrastructure,
where xref or etags or any similar feature will have to rely on the
user for telling them which database of symbols to use.

Is xref ready for these use cases?  If so, in what form (simple
variable customizations, specified commands, Lisp code that the use
must supply, something else) can the user specify what she wants?

> Would it not depend on major mode at all, so it would also be true
> in help-mode and similar buffers?

Sometimes, I guess.  Like doing that in *scratch* after "emacs -Q", or
in the *Help* buffer (which sometimes refers to external library
functions).

> If you want it in all major modes, you can use find-file-hook instead of 
> emacs-lisp-mode-hook. If in all files everywhere, then you can also drop 
> the buffer-file-name check, ending up with
> 
> (add-hook 'find-file-hook #'xref-etags-mode t)
> 
> That doesn't help with non-file buffers, though. But you can use a 
> separate major mode hook for each.

Is it possible to turn on xref-etags-mode (or its equivalents)
globally?  Are there any disadvantages of that?  IOW, why does it need
to be turned on by a hook?

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/23/2015 11:03 AM, Eli Zaretskii wrote:

> Most, if not all, projects I work on aren't "projects" in your sense
> of the word, I think -- they lack any "project" files except the
> sources and Makefile's.

A top Makefile is enough for me, conceptually.

> With etags, I switch "projects" by invoking
> visit-tags-table and typing a name of a different TAGS file (I'm then
> given a choice of whether to keep the previous TAGS table or discard
> it, which is important -- see below).

I would consider this you adapting to Emacs's lack of support for 
projects: many people like to have commands that work on the "current 
project" (find-file, replace, etc), and depend on which buffer you're 
currently in. But anyway...

> If xref can reliably deduce that I switched projects and automatically
> update its database, that's fine with me, and would probably
> constitute what you mean by "dependence on the current file or
> project".

There's no single database. If you're using the etags backend, it should 
be just as affected by `M-x visit-tags-table', as `find-tag' is. So that 
should be good enough.

> Take, for example, the use case where I'm testing a program and found
> a bug.  I then need to be able to quickly find and examine the
> definition of symbols that might be involved in the bug, look at their
> code, perhaps make some changes -- this all will be served well by
> using the database (such as TAGS) of that single program.  But suppose
> I now come to the conclusion that the bug is not in the program per
> se, but involves one of the external libraries it uses.  Now I need to
> go through the sources of that library (whose sources, by sheer luck
> or maybe something else, I already have available on my system).  How
> would xref or etags know whether I switched to that library as part of
> my previous work (and therefore still need access to the previous
> project's symbols), or because I'm now working on an entirely
> different project?  What if investigating the bug needs to
> intermittently look at the sources of the program and the library
> (e.g., because the bug happens due to some incompatibility between
> them)?

Honestly, I would probably run two instances of Emacs. But the 
tags-file-name/tags-table-list approach should still work, as long as 
the etags backend is used.

Note that currently the only thing that depends on a specific buffer is 
the backend used. And the only buffers that use the non-default value 
now are in emacs-lisp-mode or its derivatives.

Until that changes, applying either of the code samples I presented 
should give you find-tag-ish behavior everywhere. So I was a bit too 
alarmist about you having to use find-file-hook and different major mode 
hooks.... yet.

> So I think there will always be a need for asking the user about this,
> and in addition there are projects without "project" infrastructure,
> where xref or etags or any similar feature will have to rely on the
> user for telling them which database of symbols to use.

A project infrastructure should take priority over this. For instance, 
how the current Projectile users switch to a file in a "different 
project"? They call `M-x projectile-switch-project', pick one from the 
cached list of projects, and then `M-x projectile-find-file' in the 
resulting buffer. I believe this kind of workflow scales better.

The logic behind determining the current project is entirely 
configurable, so there's no reason why a directory with a Makefile can't 
be considered a project, if the user is so inclined.

> Is xref ready for these use cases?  If so, in what form (simple
> variable customizations, specified commands, Lisp code that the use
> must supply, something else) can the user specify what she wants?

If they really want to, using simple functions and major mode hooks. 
Maybe minor modes. As you've seen, the code to "get back to using etags" 
is trivial.

>> Would it not depend on major mode at all, so it would also be true
>> in help-mode and similar buffers?
>
> Sometimes, I guess.  Like doing that in *scratch* after "emacs -Q", or
> in the *Help* buffer (which sometimes refers to external library
> functions).

Right. You'll only need to worry about help-mode buffers after someone 
writes dedicated xref support for it, and if you want to ignore their 
effors (just like it seems elisp-xref-find is not valuable for you).

> Is it possible to turn on xref-etags-mode (or its equivalents)
> globally?

It's "turned on" by default. All it does is restore the default values 
of two variables, to get back to the default behavior in emacs-lisp-mode 
buffers.

(I don't understand why this is not blindingly obvious from looking at 
those several lines.)

> Are there any disadvantages of that?

The find-func package is generally considered better and easier for 
Emacs Lisp navigation than etags. It's more precise, it distinguishes 
between different kinds of symbols, AND it can easily visit definitions 
from third-party code loaded in user's Emacs.

The one downside (it won't offer non-autoloaded symbols from not loaded 
packages) doesn't seems to be a real problem in practice.

Compared to etags, it also won't jump to the C functions that are not 
exposed to Emacs Lisp. The overwhelming majority of our users (myself 
probably included), won't ever need this.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Fri, 23 Jan 2015 19:34:45 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 01/23/2015 11:03 AM, Eli Zaretskii wrote:
> 
> > Most, if not all, projects I work on aren't "projects" in your sense
> > of the word, I think -- they lack any "project" files except the
> > sources and Makefile's.
> 
> A top Makefile is enough for me, conceptually.

I didn't mean conceptually, I meant in practice.  Collecting all the
bits and pieces of a project scattered across many Makefile's in a
large tree is not an easy task.  It's little wonder most if not all
"project systems" use a single file to describe the entire project
workspace.

> > With etags, I switch "projects" by invoking
> > visit-tags-table and typing a name of a different TAGS file (I'm then
> > given a choice of whether to keep the previous TAGS table or discard
> > it, which is important -- see below).
> 
> I would consider this you adapting to Emacs's lack of support for 
> projects: many people like to have commands that work on the "current 
> project" (find-file, replace, etc), and depend on which buffer you're 
> currently in. But anyway...

I also like commands that work on the current project, I just don't
like them to depend too much on the current buffer.  E.g., I have a
habit of switching to *scratch* quite often and leave random notes
there, or text I'd like to have at my fingertips.  That doesn't mean
that when I'm in *scratch* I'm not working on the same project I was a
minute ago and need to be able to access the same symbols.

IOW, I find it hard to believe Emacs can be smart enough to know when
I switch projects; at best it could employ some heuristics to offer me
that for confirmation.  Realistically, we need a command to switch to
another project, and xref should plug into it, instead of trying to
second-guess what I want.

> > If xref can reliably deduce that I switched projects and automatically
> > update its database, that's fine with me, and would probably
> > constitute what you mean by "dependence on the current file or
> > project".
> 
> There's no single database.

It doesn't matter if there's one or many.  The important part is that
the active one(s) need to be switched when I change to another
project, and otherwise kept until I say-so.

> If you're using the etags backend, it should be just as affected by
> `M-x visit-tags-table', as `find-tag' is.

I don't understand what that means, sorry.  find-tag cannot work
without me visiting a TAGS table.

> > Take, for example, the use case where I'm testing a program and found
> > a bug.  I then need to be able to quickly find and examine the
> > definition of symbols that might be involved in the bug, look at their
> > code, perhaps make some changes -- this all will be served well by
> > using the database (such as TAGS) of that single program.  But suppose
> > I now come to the conclusion that the bug is not in the program per
> > se, but involves one of the external libraries it uses.  Now I need to
> > go through the sources of that library (whose sources, by sheer luck
> > or maybe something else, I already have available on my system).  How
> > would xref or etags know whether I switched to that library as part of
> > my previous work (and therefore still need access to the previous
> > project's symbols), or because I'm now working on an entirely
> > different project?  What if investigating the bug needs to
> > intermittently look at the sources of the program and the library
> > (e.g., because the bug happens due to some incompatibility between
> > them)?
> 
> Honestly, I would probably run two instances of Emacs.

Blasphemy ;-)

Seriously, though: I very much hope this was a semi-joke, because we
really don't want asking users to do that.  A single instance of Emacs
should be able to cope with such a simple task.  FWIW, I regularly
have about a dozen projects in my Emacs session, and switch between
them quite a lot.

> But the tags-file-name/tags-table-list approach should still work,
> as long as the etags backend is used.

I thought you were asking whether I must stick to etags, and was
trying to explain which features I need to have, not necessarily in
etags.  If the only conclusion from this discussion is that I must
stick to etags or give up on features that it has been offering for
years, then I'm sorry to say that xref doesn't seem progress to me.

> Note that currently the only thing that depends on a specific buffer is 
> the backend used.

I fail to understand the rationale behind such a design.  As long as
we are talking about buffers that hold source code of some programming
language, why would it be a good idea to switch backends depending on
the buffer?

> > So I think there will always be a need for asking the user about this,
> > and in addition there are projects without "project" infrastructure,
> > where xref or etags or any similar feature will have to rely on the
> > user for telling them which database of symbols to use.
> 
> A project infrastructure should take priority over this. For instance, 
> how the current Projectile users switch to a file in a "different 
> project"? They call `M-x projectile-switch-project', pick one from the 
> cached list of projects, and then `M-x projectile-find-file' in the 
> resulting buffer. I believe this kind of workflow scales better.

projectile-switch-project is conceptually the same as
visit-tags-table, so I see no difference.  Not sure what kind of
scaling is being alluded to here; in any case, I wasn't talking about
finding files, although TAGS allow that as well (I never used that
feature of etags).

> The logic behind determining the current project is entirely 
> configurable, so there's no reason why a directory with a Makefile can't 
> be considered a project, if the user is so inclined.

That's not the issue.  The issue is that the logic behind determining
the current project cannot be reliable enough to decide for the user
which collection(s) of symbols are of interest to me at any given
time.  Only the user knows that.  Unlike some other heuristics, where
it's okay to have an imperfect success rate, in this case it's
terribly annoying when Emacs refuses to admit that a symbol exists and
show it to you, when you _know_ for sure it does exist.  This kind of
annoyance _will_ happen if you rely on automated logic for deciding
which symbols are or aren't relevant.

> > Is xref ready for these use cases?  If so, in what form (simple
> > variable customizations, specified commands, Lisp code that the use
> > must supply, something else) can the user specify what she wants?
> 
> If they really want to, using simple functions and major mode hooks. 
> Maybe minor modes. As you've seen, the code to "get back to using etags" 
> is trivial.

That's not user-level customization.  It's too complex to be that.  We
need simpler, higher-level customizations.

> > Is it possible to turn on xref-etags-mode (or its equivalents)
> > globally?
> 
> It's "turned on" by default.

Then why did you propose to use hooks?

> Compared to etags, it also won't jump to the C functions that are not 
> exposed to Emacs Lisp. The overwhelming majority of our users (myself 
> probably included), won't ever need this.

You can't build useful features on such assumptions.

Btw, this mode of work, in more than one language, is not limited to
Emacs.  You will see it in Guile, in Python, in Awk, in GDB, and in
many other places.  It begins to be the rule rather than exception
these days.  It's not good if Emacs will ask developers to jump
through hoops to support that.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Stefan Monnier]

>> If xref can reliably deduce that I switched projects and automatically
>> update its database, that's fine with me, and would probably
>> constitute what you mean by "dependence on the current file or
>> project".

Right, ideally that's what tags.el should do: it should associate
a "file-system area" to each TAGS file; so it can know automatically
which TAGS file to use based on default-directory.

Whether it then keeps several TAGS file opened at the same time, or
whether it only keeps a single-one-at-a-time (and reloads the other
when you move to a buffer that belongs to a different project) is just
an implementation detail.

>> Take, for example, the use case where I'm testing a program and found
>> a bug.  I then need to be able to quickly find and examine the
>> definition of symbols that might be involved in the bug, look at their
>> code, perhaps make some changes -- this all will be served well by
>> using the database (such as TAGS) of that single program.  But suppose
>> I now come to the conclusion that the bug is not in the program per
>> se, but involves one of the external libraries it uses.  Now I need to
>> go through the sources of that library (whose sources, by sheer luck
>> or maybe something else, I already have available on my system).  How
>> would xref or etags know whether I switched to that library as part of
>> my previous work (and therefore still need access to the previous
>> project's symbols), or because I'm now working on an entirely
>> different project?

It wouldn't and it would only give you access to the identifiers of the
project to which the current-buffer belongs.  So if you use M-. from
a file in the buggy library it's use the TAGS file of that library, and
if you then go to a different buffer belonging to your program and use
M-. xref should then use the TAGS file of that program.  I.e. you tell
Emacs which TAGS file to use by selecting an appropriate buffer before
you hit M-.


        Stefan

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  eller.helmut@gmail.com,  19466@debbugs.gnu.org
> Date: Fri, 23 Jan 2015 16:15:49 -0500
> 
> >> If xref can reliably deduce that I switched projects and automatically
> >> update its database, that's fine with me, and would probably
> >> constitute what you mean by "dependence on the current file or
> >> project".
> 
> Right, ideally that's what tags.el should do: it should associate
> a "file-system area" to each TAGS file; so it can know automatically
> which TAGS file to use based on default-directory.

It does (try "M-x visit-tags-table" and you will see that the default
it suggests is like you say).  It just doesn't automatically load
TAGS, because the fact that I am in some buffer doesn't yet mean that
I'm working on that buffer's project and need its TAGS.

> Whether it then keeps several TAGS file opened at the same time, or
> whether it only keeps a single-one-at-a-time (and reloads the other
> when you move to a buffer that belongs to a different project) is just
> an implementation detail.

The important part is that it allows me to keep or discard portions of
the tags table.  _That_ is certainly NOT an implementation detail.

> >> Take, for example, the use case where I'm testing a program and found
> >> a bug.  I then need to be able to quickly find and examine the
> >> definition of symbols that might be involved in the bug, look at their
> >> code, perhaps make some changes -- this all will be served well by
> >> using the database (such as TAGS) of that single program.  But suppose
> >> I now come to the conclusion that the bug is not in the program per
> >> se, but involves one of the external libraries it uses.  Now I need to
> >> go through the sources of that library (whose sources, by sheer luck
> >> or maybe something else, I already have available on my system).  How
> >> would xref or etags know whether I switched to that library as part of
> >> my previous work (and therefore still need access to the previous
> >> project's symbols), or because I'm now working on an entirely
> >> different project?
> 
> It wouldn't and it would only give you access to the identifiers of the
> project to which the current-buffer belongs.  So if you use M-. from
> a file in the buggy library it's use the TAGS file of that library, and
> if you then go to a different buffer belonging to your program and use
> M-. xref should then use the TAGS file of that program.  I.e. you tell
> Emacs which TAGS file to use by selecting an appropriate buffer before
> you hit M-.

So you are saying that when I'm in some buffer, I cannot look at some
of the identifiers until I switch to another buffer?  That's very
inconvenient.  Again, when I work on a problem that happens between a
program and a library, I need to see both sides of the interface at
the same time, independently of which buffer I am in.  Conceptually,
the set of identifiers that are relevant to me in this situation is
the union of both projects, and I should be able to find any
identifier from this union no matter which buffer is the current one.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/23/2015 11:03 PM, Eli Zaretskii wrote:

> I also like commands that work on the current project, I just don't
> like them to depend too much on the current buffer.

To cut this part of discussion short, yes, the project system can well 
be designed so that the "current project" depends not on the current 
buffer but on the user's choice. So to switch between projects, you 
invoke an explicit command, but don't visit a buffer belonging to the 
other project. And xref can work with such a system (just as it works 
with etags) without significant problems.

> Realistically, we need a command to switch to
> another project, and xref should plug into it, instead of trying to
> second-guess what I want.

Instead of xref plugging into whatever, any project system (which I'm 
assuming is implemented as a minor mode) can plug into xref and set up 
xref-find-function, etc, to use itself.

> It doesn't matter if there's one or many.  The important part is that
> the active one(s) need to be switched when I change to another
> project, and otherwise kept until I say-so.

That depends on the backend's implementation. etags works that way.

>> If you're using the etags backend, it should be just as affected by
>> `M-x visit-tags-table', as `find-tag' is.
>
> I don't understand what that means, sorry.  find-tag cannot work
> without me visiting a TAGS table.

etags-xref-find, likewise, will ask you for the path to the TAGS file, 
if it's not set yet.

> Seriously, though: I very much hope this was a semi-joke, because we
> really don't want asking users to do that.  A single instance of Emacs
> should be able to cope with such a simple task.  FWIW, I regularly
> have about a dozen projects in my Emacs session, and switch between
> them quite a lot.

Semi-joke, yes. That's just what I often do to avoid the clutter of 
buffers opened from different projects. It could work almost as well 
using different frames, but I'd have to find a way to hide the one 
frame's file buffers from the other frames.

You can continue doing what you've always done, no problem.

>> But the tags-file-name/tags-table-list approach should still work,
>> as long as the etags backend is used.
>
> I thought you were asking whether I must stick to etags, and was
> trying to explain which features I need to have, not necessarily in
> etags.  If the only conclusion from this discussion is that I must
> stick to etags or give up on features that it has been offering for
> years, then I'm sorry to say that xref doesn't seem progress to me.

Not at all. When I say "etags backend", I mean "etags backend for xref", 
since the latter is the subject of our discussion.

I think it's high time you've tried the snippets I wrote.

>> Note that currently the only thing that depends on a specific buffer is
>> the backend used.
>
> I fail to understand the rationale behind such a design.  As long as
> we are talking about buffers that hold source code of some programming
> language, why would it be a good idea to switch backends depending on
> the buffer?

You may not like it, but it's the easiest viable approach we can take, 
and a lot of people find it natural. For instance, both original 
proposals for the xref implementation used it.

However, like I mentioned, when a project system is used, we can use its 
backend in all project files, and so the backend would depend on the 
directory you in. But also see my first paragraph in this email.

> projectile-switch-project is conceptually the same as
> visit-tags-table, so I see no difference.

Except it visits the "root file" of the project after switching, and 
immediately "forgets" the previous project. So it's the kind of design 
you don't like.

It's the most popular project solution for Emacs that I know.

> That's not the issue.  The issue is that the logic behind determining
> the current project cannot be reliable enough to decide for the user
> which collection(s) of symbols are of interest to me at any given
> time.  Only the user knows that.  Unlike some other heuristics, where
> it's okay to have an imperfect success rate, in this case it's
> terribly annoying when Emacs refuses to admit that a symbol exists and
> show it to you, when you _know_ for sure it does exist.  This kind of
> annoyance _will_ happen if you rely on automated logic for deciding
> which symbols are or aren't relevant.

While I can understand it, that's just your (i.e. not universal) 
opinion, and I believe it's addressed above. That is, a backend can use 
this approach as well.

> That's not user-level customization.  It's too complex to be that.  We
> need simpler, higher-level customizations.

We don't even know what and how to customize yet. It might be time for 
you to swap the user shoes for the developer shoes.

>>> Is it possible to turn on xref-etags-mode (or its equivalents)
>>> globally?
>>
>> It's "turned on" by default.
>
> Then why did you propose to use hooks?

Because emacs-lisp-mode overrides it, and to reach certain level of 
happiness, you seem to need to revert that override. I believe I've 
explained that in the previous email, no?

>> Compared to etags, it also won't jump to the C functions that are not
>> exposed to Emacs Lisp. The overwhelming majority of our users (myself
>> probably included), won't ever need this.
>
> You can't build useful features on such assumptions.

I can, and I do.

> Btw, this mode of work, in more than one language, is not limited to
> Emacs.  You will see it in Guile, in Python, in Awk, in GDB, and in
> many other places.  It begins to be the rule rather than exception
> these days.  It's not good if Emacs will ask developers to jump
> through hoops to support that.

You seem to be clamoring for project support in Emacs. That's 
commendable, and it's a separate discussion.

Meanwhile, a lot of features currently depend just on the current major 
mode and buffer. There's nothing new about it. The easiest example is 
code completion.

In emacs-lisp-mode, it will parse the current buffer context and offer 
completions from obarray, somewhat filtered. If you have a README in the 
same directory as foo.el (so we might consider them to be in the same 
project), when you open it, you definitely won't get the same 
completions as in emacs-lisp-mode. In fact, if there's no current tags 
file visited, in the default Emacs configuration you won't get any 
completions at all.

Similarly for python-mode (python-shell-completion-at-point only works 
in Python buffers, and only when an inferior shell is running). Likewise 
for Octave, and I just haven't looked at Guile, Awk and GDB support.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Sat, 24 Jan 2015 00:23:24 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: eller.helmut@gmail.com, 19466@debbugs.gnu.org
> 
> On 01/23/2015 11:03 PM, Eli Zaretskii wrote:
> 
> > I also like commands that work on the current project, I just don't
> > like them to depend too much on the current buffer.
> 
> To cut this part of discussion short, yes, the project system can well 
> be designed so that the "current project" depends not on the current 
> buffer but on the user's choice. So to switch between projects, you 
> invoke an explicit command, but don't visit a buffer belonging to the 
> other project. And xref can work with such a system (just as it works 
> with etags) without significant problems.

Can you tell how can this be set up with xref, or point me to the
relevant documentation?

> > Realistically, we need a command to switch to
> > another project, and xref should plug into it, instead of trying to
> > second-guess what I want.
> 
> Instead of xref plugging into whatever, any project system (which I'm 
> assuming is implemented as a minor mode) can plug into xref and set up 
> xref-find-function, etc, to use itself.

Any of these two will do (a plug has 2 parts).

> > It doesn't matter if there's one or many.  The important part is that
> > the active one(s) need to be switched when I change to another
> > project, and otherwise kept until I say-so.
> 
> That depends on the backend's implementation. etags works that way.

That's not my point.  My point is that it is up to the user to decide
when she switches projects, and tell Emacs about that.  Emacs should
not try doing that on its own, whatever the backend's implementation
is.  The backend cannot make this decision for the user.

> I think it's high time you've tried the snippets I wrote.

I said I will -- when I have time.  I didn't have that time yet,
sorry.

> >> Note that currently the only thing that depends on a specific buffer is
> >> the backend used.
> >
> > I fail to understand the rationale behind such a design.  As long as
> > we are talking about buffers that hold source code of some programming
> > language, why would it be a good idea to switch backends depending on
> > the buffer?
> 
> You may not like it, but it's the easiest viable approach we can take, 
> and a lot of people find it natural. For instance, both original 
> proposals for the xref implementation used it.

I believe you it's the easiest approach.  But that doesn't yet make it
viable, because IMO it doesn't scale.  Only very simple projects will
be happy with such a design.  The example I described is not very
complicated, either, but it already bumps into this limitation.  I
think this is a clear sign that the limitation should be lifted.

> However, like I mentioned, when a project system is used, we can use its 
> backend in all project files, and so the backend would depend on the 
> directory you in.

If lifting the above limitation requires some project system, then we
should provide a simple one for a project that uses only Makefile's.
That could be a single command that just specifies the top-level
directory of the project, or something similar.

However, it sounds like xref is not yet ready for that, either, is it?
If so, I suggest to add these simple features, because that would
allow to lift the limitation and make the feature much more scalable.

> > projectile-switch-project is conceptually the same as
> > visit-tags-table, so I see no difference.
> 
> Except it visits the "root file" of the project after switching, and 
> immediately "forgets" the previous project. So it's the kind of design 
> you don't like.

Then there should be a projectile-add-project and
projectile-remove-project as well.  I'm sure someone already thought
about that.  (I know nothing about Projectile, except that I think the
name is unfortunate -- take this from someone who has an intimate
familiarity with real projectiles.)

> > That's not the issue.  The issue is that the logic behind determining
> > the current project cannot be reliable enough to decide for the user
> > which collection(s) of symbols are of interest to me at any given
> > time.  Only the user knows that.  Unlike some other heuristics, where
> > it's okay to have an imperfect success rate, in this case it's
> > terribly annoying when Emacs refuses to admit that a symbol exists and
> > show it to you, when you _know_ for sure it does exist.  This kind of
> > annoyance _will_ happen if you rely on automated logic for deciding
> > which symbols are or aren't relevant.
> 
> While I can understand it, that's just your (i.e. not universal) 
> opinion

I'd be surprised if it was just mine.  I know people who work on
projects much more complex than the use case I described.  Perhaps
they don't use Emacs, which would be an indirect approval of my
opinions.

> That is, a backend can use this approach as well.

"Can" and "does" have a large gap between them, from the user's POV.
Do we already have such a backend?  If not, I suggest to add it.

> > That's not user-level customization.  It's too complex to be that.  We
> > need simpler, higher-level customizations.
> 
> We don't even know what and how to customize yet. It might be time for 
> you to swap the user shoes for the developer shoes.

Unlikely to happen.  Given that etags exists and pretty much satisfies
my needs (with 30-year old technology!), there's not enough itch here
for me to scratch.  If xref is going to be as good as etags, let alone
better, I _will_ switch, I can promise you that much.  Until then, I'm
happy enough with etags, thank you very much.  I'm here merely as a
user, providing feedback for features someone else develops.  I hope
this is helpful; if not, feel free to discontinue the discussion at
any point you like.

> >>> Is it possible to turn on xref-etags-mode (or its equivalents)
> >>> globally?
> >>
> >> It's "turned on" by default.
> >
> > Then why did you propose to use hooks?
> 
> Because emacs-lisp-mode overrides it, and to reach certain level of 
> happiness, you seem to need to revert that override. I believe I've 
> explained that in the previous email, no?

No, I only understood that now.  Sorry for being dense.

So perhaps some users, like myself, will benefit from preventing the
override?  Is that possible with the current master?

> > Btw, this mode of work, in more than one language, is not limited to
> > Emacs.  You will see it in Guile, in Python, in Awk, in GDB, and in
> > many other places.  It begins to be the rule rather than exception
> > these days.  It's not good if Emacs will ask developers to jump
> > through hoops to support that.
> 
> You seem to be clamoring for project support in Emacs.

Maybe I am, I don't know.  I don't care how this is called.  What I do
care is to be able to start working on a project with minimal fuss.
Having to spend a significant time telling Emacs just what constitutes
the project is therefore a turn-off: it is why I dislike Visual Studio
like "projects" and "solutions" system -- it might be OK for starting
a project from scratch, but sucks when you need to work on an existing
project, let alone a large one.

> That's commendable, and it's a separate discussion.

It is a very much related discussion, since you think the features I
miss in xref should be implemented by "project support".  Because
etags seems to seamlessly let me have that.  So if xref is meant to
replace etags, it should provide the same functionality, and if that
requires some rudimentary "project support", we should include it when
we announce deprecation of etags.

> Meanwhile, a lot of features currently depend just on the current major 
> mode and buffer. There's nothing new about it.

I never said that no feature can ever depend on the major mode.  All
I'm saying is that this particular feature shouldn't, and I gave
simple enough use cases to demonstrate why it is IMO wrong in this
case.

> The easiest example is code completion.  In emacs-lisp-mode, it will
> parse the current buffer context and offer completions from obarray,
> somewhat filtered. If you have a README in the same directory as
> foo.el (so we might consider them to be in the same project), when
> you open it, you definitely won't get the same completions as in
> emacs-lisp-mode. In fact, if there's no current tags file visited,
> in the default Emacs configuration you won't get any completions at
> all.
> 
> Similarly for python-mode (python-shell-completion-at-point only works 
> in Python buffers, and only when an inferior shell is running). Likewise 
> for Octave, and I just haven't looked at Guile, Awk and GDB support.

I'm not sure what completions are alluded to here.  Completions are
context-dependent, i.e. their DWIM-ish nature should depend on what is
being completed, and for what purposes.  For example, I hope you'd
agree that it would be bad in general for completion in "C-x C-f" to
show only files from the current project.  By contrast, a specialized
command that is meant to find files only from the current project
should probably limit the candidates to the project.

Likewise with symbols -- when the user wants only symbols from the
current project, or from the programming language of the current
buffer, that's what Emacs should show.  But there are also situations
where such limitations produce bad results.

Since only the user knows what she wants to see at any given moment,
Emacs must support that.  It can do that in several ways:

  . have dedicated commands for each kind of situation

  . apply some heuristics to order the candidates in a particular way,
    so that the ones deemed more DWIM-ish appear first; this
    heuristics could use the current major mode or the current project
    or something else

  . maybe some other possibilities I haven't thought about

The important point is we cannot limit the user to just one possible
context, even if it is the most frequent one.  Such a feature would be
too limited.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/24/2015 11:40 AM, Eli Zaretskii wrote:

>> To cut this part of discussion short, yes, the project system can well
>> be designed so that the "current project" depends not on the current
>> buffer but on the user's choice. So to switch between projects, you
>> invoke an explicit command, but don't visit a buffer belonging to the
>> other project. And xref can work with such a system (just as it works
>> with etags) without significant problems.
>
> Can you tell how can this be set up with xref, or point me to the
> relevant documentation?

I did. "Just like it works with etags" was a big hint in the quote above.

> I said I will -- when I have time.  I didn't have that time yet,
> sorry.

Please respond after you've found that time, then. Is it is, the current 
discussion is getting nowhere.

> I believe you it's the easiest approach.  But that doesn't yet make it
> viable, because IMO it doesn't scale.  Only very simple projects will
> be happy with such a design.  The example I described is not very
> complicated, either, but it already bumps into this limitation.  I
> think this is a clear sign that the limitation should be lifted.

Just for the sake of the argument: instead of visiting two tag tables at 
the same time from Emacs, you could instead include the external 
library's tag file in the application's tag file ("--include" etags 
argument). That doesn't seem like it'll work with several external 
libraries, but maybe it should.

That wouldn't allow you to jump from a library file to the application's 
file (I think; you'd have to use e.g. switch-buffer instead), but there 
is a certain semantic strictness in that (the library doesn't know about 
the application).

> However, it sounds like xref is not yet ready for that, either, is it?
> If so, I suggest to add these simple features, because that would
> allow to lift the limitation and make the feature much more scalable.

It's quite ready. xref will work with any such backend.

> Then there should be a projectile-add-project and
> projectile-remove-project as well.  I'm sure someone already thought
> about that.

Maybe they didn't but they haven't shared that with the rest of us. 
Given Projectile's current behavior, though it would cause 
incompatibility or complications in the implementation.

But anyway, I think I'm starting to like the idea of this approach. 
Maybe I'll try to implement it sometime.

> (I know nothing about Projectile, except that I think the
> name is unfortunate -- take this from someone who has an intimate
> familiarity with real projectiles.)

I think it's fine. A projectile is not necessarily an ammo round.

>> That is, a backend can use this approach as well.
>
> "Can" and "does" have a large gap between them, from the user's POV.
> Do we already have such a backend?  If not, I suggest to add it.

etagggggsss.

Its only drawback, from your standpoint, stems from it being the default 
one (which major modes can override), instead from being assigned in a 
globalized minor mode.

I'm not sure if we want to have it both as default, and have a minor 
mode (a by-default-on minor mode won't be good enough).

 > I'm here merely as a user, providing feedback for features someone
 > else develops.  I hope this is helpful; if not, feel free to
 > discontinue the discussion at any point you like.

I you were just a user, I'd have stopped a while ago, since I still 
don't think this feature request is useful to the majority of users.

Since you're a valued core contributor, though, I'd really like to 
accommodate your needs. However, that comes with the expectation that 
you can participate in the design and provide more nuanced requirements 
than an ordinary user would do. Maybe you'd a least look at the code, if 
not implement the improvements yourself.

> So perhaps some users, like myself, will benefit from preventing the
> override?  Is that possible with the current master?

Yes: you use the snippets I provided together with the current master. 
Whether any part of them is worthy of installing into master, is yet to 
be determined.

> Maybe I am, I don't know.  I don't care how this is called.  What I do
> care is to be able to start working on a project with minimal fuss.
> Having to spend a significant time telling Emacs just what constitutes
> the project is therefore a turn-off: it is why I dislike Visual Studio
> like "projects" and "solutions" system -- it might be OK for starting
> a project from scratch, but sucks when you need to work on an existing
> project, let alone a large one.

I'm pretty sure some users would consider having to call 
visit-tags-table as "having to spend time telling Emacs just what 
constitutes a project", as opposed to Projectile's current approach.

> It is a very much related discussion, since you think the features I
> miss in xref should be implemented by "project support".

No: the xref features should work with etags without additional 
infrastructure. We won't know that for certain, though, until you find 
that free time.

I mentioned project support, which we don't have, because you've 
expanded the discussion to supporting "this mode of work" in different 
languages and, I'm assuming, for different features. And not every 
language/framework ecosystem considers the tag files to be the best 
solution for code navigation, completion, etc.

I'll respond to the rest of this email in a separate thread, but that 
discussion is likely premature.

> etags seems to seamlessly let me have that.  So if xref is meant to
> replace etags, it should provide the same functionality, and if that
> requires some rudimentary "project support", we should include it when
> we announce deprecation of etags.

xref is meant to replace `find-tag' and `tags-apropos'. etags, as a 
package, is unlikely to ever be deprecated.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Sat, 24 Jan 2015 18:47:46 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> > I said I will -- when I have time.  I didn't have that time yet,
> > sorry.
> 
> Please respond after you've found that time, then. Is it is, the current 
> discussion is getting nowhere.

I tried that a few days ago, but didn't see any significant changes in
behavior.  I'm probably missing something -- what exactly did you want
me to pay attention to?

Thanks.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 01/31/2015 10:52 AM, Eli Zaretskii wrote:

> I tried that a few days ago, but didn't see any significant changes in
> behavior.  I'm probably missing something -- what exactly did you want
> me to pay attention to?

What have you tried, exactly?

You should have noticed that `M-.' in emacs-lisp-mode buffers behaves 
like in other buffers and uses the current tags table (and prompts for 
it if the tags table hasn't been visited yet).

I've found one caveat now: even though the tags list is not buffer-local 
(right?), (tags-lazy-completion-table) returns different results in 
lisp/**/*.el buffers and src/*.c buffers.

`find-tag' completion exhibits the same difference. For instance, 
calling `M-x find-tag' in src/disp.c, then typing `display_li' and 
pressing TAB will complete it to `display_line'. No so in 
lisp/progmodes/etags.el. Doing it in that buffer results in [No match]. 
However, typing `display_line' fully in either, then pressing RET, 
brings you to that function's definition. This should be considered a 
bug, right?

Until it is fixed, to be able to jump to `display_line' from lisp/** 
buffers, we have to disable the strict matching in 
`xref--read-identifier'. Please try #41 again with this patch applied:

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index 55405b6..f550282 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -562,7 +562,7 @@ Return an alist of the form ((FILENAME . (XREF ...)) 
...)."
      (cond ((or current-prefix-arg (not id))
             (completing-read prompt
                              (funcall 
xref-identifier-completion-table-function)
-                            nil t id
+                            nil nil id
                              'xref--read-identifier-history))
            (t id))))

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 04:24:50 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 01/31/2015 10:52 AM, Eli Zaretskii wrote:
> 
> > I tried that a few days ago, but didn't see any significant changes in
> > behavior.  I'm probably missing something -- what exactly did you want
> > me to pay attention to?
> 
> What have you tried, exactly?

I evaluated your suggested code, and then typed "M-.".

> You should have noticed that `M-.' in emacs-lisp-mode buffers behaves 
> like in other buffers and uses the current tags table (and prompts for 
> it if the tags table hasn't been visited yet).

It does.

> I've found one caveat now: even though the tags list is not buffer-local 
> (right?), (tags-lazy-completion-table) returns different results in 
> lisp/**/*.el buffers and src/*.c buffers.

Yes, it's not 100% smooth.

> `find-tag' completion exhibits the same difference. For instance, 
> calling `M-x find-tag' in src/disp.c, then typing `display_li' and 
> pressing TAB will complete it to `display_line'. No so in 
> lisp/progmodes/etags.el. Doing it in that buffer results in [No match]. 

That's not what I see, both in Emacs 24.4 and with the current trunk:
the completion works even in buffers whose major mode is emacs-lisp.

> However, typing `display_line' fully in either, then pressing RET, 
> brings you to that function's definition. This should be considered a 
> bug, right?

Yes, except that I don't see it, at least not in "emacs -Q".

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 02/01/2015 06:01 PM, Eli Zaretskii wrote:

>> What have you tried, exactly?
>
> I evaluated your suggested code, and then typed "M-.".

If you recall, I gave you several options, with the end goal being 
making the choice between them or maybe something else.

So it would help if you mentioned exactly which one you tried and the 
problems you experienced after.

>> You should have noticed that `M-.' in emacs-lisp-mode buffers behaves
>> like in other buffers and uses the current tags table (and prompts for
>> it if the tags table hasn't been visited yet).
>
> It does.

Good. So why did you report not seeing any significant changes?

>> I've found one caveat now: even though the tags list is not buffer-local
>> (right?), (tags-lazy-completion-table) returns different results in
>> lisp/**/*.el buffers and src/*.c buffers.
>
> Yes, it's not 100% smooth.

What do you mean by that exactly?

It can't be that same as what I meant because you've rejected the only 
example I gave.

>> `find-tag' completion exhibits the same difference. For instance,
>> calling `M-x find-tag' in src/disp.c, then typing `display_li' and
>> pressing TAB will complete it to `display_line'. No so in
>> lisp/progmodes/etags.el. Doing it in that buffer results in [No match].
>
> That's not what I see, both in Emacs 24.4 and with the current trunk:
> the completion works even in buffers whose major mode is emacs-lisp.

Good to know. That means I did something unusual, and the average user 
won't see that problem.

>> However, typing `display_line' fully in either, then pressing RET,
>> brings you to that function's definition. This should be considered a
>> bug, right?
>
> Yes, except that I don't see it, at least not in "emacs -Q".

This has nothing to do with my configuration, just with a few (maybe 
ill-considered) steps that I performed: http://debbugs.gnu.org/19741

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 22:11:08 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 02/01/2015 06:01 PM, Eli Zaretskii wrote:
> 
> >> What have you tried, exactly?
> >
> > I evaluated your suggested code, and then typed "M-.".
> 
> If you recall, I gave you several options, with the end goal being 
> making the choice between them or maybe something else.
> 
> So it would help if you mentioned exactly which one you tried and the 
> problems you experienced after.

They all did the same, AFAICS.

> >> You should have noticed that `M-.' in emacs-lisp-mode buffers behaves
> >> like in other buffers and uses the current tags table (and prompts for
> >> it if the tags table hasn't been visited yet).
> >
> > It does.
> 
> Good. So why did you report not seeing any significant changes?

Because I thought the effects were supposed to be more profound than
that.

> >> I've found one caveat now: even though the tags list is not buffer-local
> >> (right?), (tags-lazy-completion-table) returns different results in
> >> lisp/**/*.el buffers and src/*.c buffers.
> >
> > Yes, it's not 100% smooth.
> 
> What do you mean by that exactly?

What you discovered: the order of loading Emacs TAGS tables matters.

I have never loaded lisp/TAGS directly, only src/TAGS, which then
includes lisp/TAGS.

> It can't be that same as what I meant because you've rejected the only 
> example I gave.

I misunderstood your recipe, sorry.  I only loaded src/TAGS.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 02/01/2015 10:30 PM, Eli Zaretskii wrote:

> They all did the same, AFAICS.

The end behavior is similar, but in #32, the user can `M-x 
xref-etags-mode' again to get back to using find-func as xref backend in 
emacs-lisp-mode buffers.

> Because I thought the effects were supposed to be more profound than
> that.

It solves the original complaint, doesn't it? That's the idea.

If that's actually enough for you, we can add xref-reset-backend from 
#41 to xref.el as `xref-reset-to-etags', and anyone will be able to

(add-hook 'emacs-lisp-mode-hook 'xref-reset-to-etags)

in their init file to get back to using etags. And same with any other 
major mode that will implement its own xref backend.

> What you discovered: the order of loading Emacs TAGS tables matters.

What I see there looks like a bug, and not like a necessary quirk in 
behavior.

> I misunderstood your recipe, sorry.  I only loaded src/TAGS.

That's my fault, I didn't mention doing that in the previous message.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 22:53:11 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 19466@debbugs.gnu.org, eller.helmut@gmail.com
> 
> On 02/01/2015 10:30 PM, Eli Zaretskii wrote:
> 
> > They all did the same, AFAICS.
> 
> The end behavior is similar, but in #32, the user can `M-x 
> xref-etags-mode' again to get back to using find-func as xref backend in 
> emacs-lisp-mode buffers.

Then maybe this one is preferable.

> It solves the original complaint, doesn't it? That's the idea.
> 
> If that's actually enough for you, we can add xref-reset-backend from 
> #41 to xref.el as `xref-reset-to-etags', and anyone will be able to
> 
> (add-hook 'emacs-lisp-mode-hook 'xref-reset-to-etags)
> 
> in their init file to get back to using etags. And same with any other 
> major mode that will implement its own xref backend.

Sounds good, thanks.

bug#19466: 25.0.50; xref-find-def doesn't find C functions

[Dmitry Gutov]

On 02/02/2015 05:32 AM, Eli Zaretskii wrote:

 >> The end behavior is similar, but in #32, the user can `M-x
>> xref-etags-mode' again to get back to using find-func as xref backend in
>> emacs-lisp-mode buffers.
>
> Then maybe this one is preferable.

Added now. This bug looks resolved to me, so closing.