bug#21934: 24.5; find-tag: reading TAGS file incorrectly

[Dmitry Gutov <dgutov@yandex.ru>]

On 11/21/2015 03:07 PM, Eli Zaretskii wrote:

> Yes, they do.  Etags doesn't treat a period '.' as ending an
> identifier, except in C-like languages.  And that is good, since Lua
> evidently wants to support identifiers with embedded periods.

In that case, the submitted patch would indeed make things better. 
However, I wonder if we should rewrite the whole regexp in 
etags-tags-completion-table, to be more faithful to the "implicit tag" 
spec, and instead of whitelisting characters that can be used in tags, 
allow any character in NONAM.

(Or rather, I wonder why it hasn't been written that way to begin with, 
and whether there are some performance pitfalls in doing so).

> I've looked at the related code, and my conclusion is that there's
> more to this than meets the eye.

Indeed.

> The OP complained about _completion_ on tag names, and suggested to
> fix a regexp used by etags-tags-completion-table.  That regexp indeed
> doesn't allow a period in an identifier name (probably because it's
> disallowed in C-like languages).  However, find-tag itself doesn't use
> that regexp,

If does, for completion. Type M-x find-tag RET TAB, and you'll see the 
result of calling etags-tags-completion-table.

> so typing "M-x find-tag RET Rectangle.getPos RET" finds
> that identifier with no problems.

That's right: the logic to "list all tags" and to "find tag named xyz" 
is implemented in different places. And the latter is more faithful to 
the "implicit tag name" definition; it's the completion table logic that 
needs to be fixed.

> Now, find-tag is deprecated in Emacs 25, and M-. invokes
> xref-find-definitions instead.  AFAIU, etags-tags-completion-table is
> no longer relevant with xref.

It's (almost) just as relevant: type C-u M-. TAB.

> xref-find-definitions, if it's invoked
> with a prefix argument, and if you type "Rectangle.getPos RET" at its
> prompt, not surprisingly also finds the identifier.  Trying to invoke
> completion after "C-u M-.", with test.lua as the current buffer,
> doesn't succeed to complete even on Rectangle, so the situation here
> is somewhat worse, but I'm not sure why.

Is it really any different? M-x find-tag RET TAB doesn't show anything 
beginning with "Rectangle" either. I only get "getPos" as a completion 
either way.

> If we want "M-." without prefix argument to be able to find these
> identifiers, we need first to take care of how it determines the
> symbol at point.  Currently, it calls (thing-at-point 'symbol), which
> predictably guesses wrong.

Right, I haven't thought about it.

But what else, really, could (xref-backend-identifier-at-point 'etags) 
return here? It only knows the current buffer's syntax table, and that 
its data source is etags.

Either etags will have to consider that in both example declarations the 
tag name must be "getPos", not "xxx.getPos", and put this tag name 
explicitly into the entries, or lua-mode will have to change the syntax 
class of "." to "symbol", so that (thing-at-point 'symbol) returns 
"Circle.getPos" as the tag name.

That is, of course, if "." always goes after the name of the 
class/module/etc that "getPos" is defined in, and it can't follow a 
variable name. I'm not familiar with Lua's syntax.