bug#21934: 24.5; find-tag: reading TAGS file incorrectly

[Eli Zaretskii <eliz@gnu.org>]

> Cc: Andreas Matthias <andreas.matthias@gmail.com>
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 17 Nov 2015 06:01:27 +0200

Sorry for the delay; Life™ intervened.

> Eli, please take a look at TAGS attached to this bug report. Do the 
> entries there satisfy the "implicit name" conditions?

Yes, they do.  Etags doesn't treat a period '.' as ending an
identifier, except in C-like languages.  And that is good, since Lua
evidently wants to support identifiers with embedded periods.

> I also have a different etags in my system, that comes from Exuberant 
> Ctags, and it generates a different TAGS, looking like this:
> 
> test.lua,98
> function Rectangle.getPos ()^?Rectangle.getPos ^A2,15
> function Circle.getPos ()^?Circle.getPos ^A6,61
> 
> And that one works with `find-tag' just fine. So it would be nice if Eli 
> could comment on the difference, and whether etags should be patched 
> instead.

I've looked at the related code, and my conclusion is that there's
more to this than meets the eye.

The OP complained about _completion_ on tag names, and suggested to
fix a regexp used by etags-tags-completion-table.  That regexp indeed
doesn't allow a period in an identifier name (probably because it's
disallowed in C-like languages).  However, find-tag itself doesn't use
that regexp, so typing "M-x find-tag RET Rectangle.getPos RET" finds
that identifier with no problems.

Now, find-tag is deprecated in Emacs 25, and M-. invokes
xref-find-definitions instead.  AFAIU, etags-tags-completion-table is
no longer relevant with xref.  xref-find-definitions, if it's invoked
with a prefix argument, and if you type "Rectangle.getPos RET" at its
prompt, not surprisingly also finds the identifier.  Trying to invoke
completion after "C-u M-.", with test.lua as the current buffer,
doesn't succeed to complete even on Rectangle, so the situation here
is somewhat worse, but I'm not sure why.

If we want "M-." without prefix argument to be able to find these
identifiers, we need first to take care of how it determines the
symbol at point.  Currently, it calls (thing-at-point 'symbol), which
predictably guesses wrong.

IOW, we could fix the regexp as suggested by the OP (AFAICS, that
should not cause any regressions for etags.el), but that won't solve
the problems "M-." in Emacs 25 will have with such identifiers in Lua
sources.

Suggestions and comments are welcome.