bug#38511: etags seems to be confused by macros

bug#38511: etags seems to be confused by macros

[Skip Montanaro]

I use speedbar in Emacs, which relies on etags to generate tag files.
The Python source defines a macro named Py_LOCAL_INLINE, which is used
to select the appropriate spelling of "static inline":

#if defined(_MSC_VER)
...
#  define Py_LOCAL_INLINE(type) static __inline type __fastcall
#else
...
#  define Py_LOCAL_INLINE(type) static inline type
#endif

It's used like so:

Py_LOCAL_INLINE(void)
stackdepth_push(basicblock ***sp, basicblock *b, int depth)
{
    assert(b->b_startdepth < 0 || b->b_startdepth == depth);
    if (b->b_startdepth < depth && b->b_startdepth < 100) {
        assert(b->b_startdepth < 0);
        b->b_startdepth = depth;
        *(*sp)++ = b;
    }
}

It all works well. Etags though, thinks the file contains a function
named "Py_LOCAL_INLINE" and completely misses the actual function,
"stackdepth_push".

Seems like a bug to me, but it's not obvious if there is a trivial
fix. As a workaround, I think I can write a little shell script which
effectively expands the Py_LOCAL_INLINE macro, then pumps the result
to etags. Still, would be kind of nice if this could be fixed.

Thanks,

Skip Montanaro

bug#38511: etags seems to be confused by macros

[Eli Zaretskii]

> From: Skip Montanaro <skip.montanaro@gmail.com>
> Date: Fri, 6 Dec 2019 14:20:14 -0600
> 
> I use speedbar in Emacs, which relies on etags to generate tag files.
> The Python source defines a macro named Py_LOCAL_INLINE, which is used
> to select the appropriate spelling of "static inline":
> 
> #if defined(_MSC_VER)
> ...
> #  define Py_LOCAL_INLINE(type) static __inline type __fastcall
> #else
> ...
> #  define Py_LOCAL_INLINE(type) static inline type
> #endif
> 
> It's used like so:
> 
> Py_LOCAL_INLINE(void)
> stackdepth_push(basicblock ***sp, basicblock *b, int depth)
> {
>     assert(b->b_startdepth < 0 || b->b_startdepth == depth);
>     if (b->b_startdepth < depth && b->b_startdepth < 100) {
>         assert(b->b_startdepth < 0);
>         b->b_startdepth = depth;
>         *(*sp)++ = b;
>     }
> }
> 
> It all works well. Etags though, thinks the file contains a function
> named "Py_LOCAL_INLINE" and completely misses the actual function,
> "stackdepth_push".

Etags works by looking at the source at face value, and it doesn't
expand macros as a C compiler would.  So this:

  Py_LOCAL_INLINE(void) stackdepth_push(basicblock ***sp, basicblock *b, int depth)

looks to its naïve lexical analysis very much like a K&R definition of
a function and declaration of its arguments:

  Py_LOCAL_INLINE(foo) int foo(bar)

> Seems like a bug to me, but it's not obvious if there is a trivial
> fix.

Patches are welcome, if someone has an idea for how to fix that.

Thanks.

bug#38511: etags seems to be confused by macros

[Francesco Potortì]

>> From: Skip Montanaro <skip.montanaro@gmail.com>
>> Date: Fri, 6 Dec 2019 14:20:14 -0600
>> 
>> I use speedbar in Emacs, which relies on etags to generate tag files.
>> The Python source defines a macro named Py_LOCAL_INLINE, which is used
>> to select the appropriate spelling of "static inline":
>> 
>> #if defined(_MSC_VER)
>> ...
>> #  define Py_LOCAL_INLINE(type) static __inline type __fastcall
>> #else
>> ...
>> #  define Py_LOCAL_INLINE(type) static inline type
>> #endif
>> 
>> It's used like so:
>> 
>> Py_LOCAL_INLINE(void)
>> stackdepth_push(basicblock ***sp, basicblock *b, int depth)
>> {
>>     assert(b->b_startdepth < 0 || b->b_startdepth == depth);
>>     if (b->b_startdepth < depth && b->b_startdepth < 100) {
>>         assert(b->b_startdepth < 0);
>>         b->b_startdepth = depth;
>>         *(*sp)++ = b;
>>     }
>> }
>> 
>> It all works well. Etags though, thinks the file contains a function
>> named "Py_LOCAL_INLINE" and completely misses the actual function,
>> "stackdepth_push".

Eli Zaretskii:
>Etags works by looking at the source at face value, and it doesn't
>expand macros as a C compiler would.  So this:
>
>  Py_LOCAL_INLINE(void) stackdepth_push(basicblock ***sp, basicblock *b, int depth)
>
>looks to its naïve lexical analysis very much like a K&R definition of
>a function and declaration of its arguments:
>
>  Py_LOCAL_INLINE(foo) int foo(bar)
>
>> Seems like a bug to me, but it's not obvious if there is a trivial
>> fix.
>
>Patches are welcome, if someone has an idea for how to fix that.

Unless some C code wizard steps up and contradicts me, I'd say that it
is in principle impossible for Etags to detect such macros.  This is a
work for the --regex feature of Etags, which is thought just for this
sort of situations.

The Etags man page explains it with examples, as does the info page.

--
fp

bug#38511: etags seems to be confused by macros

[Skip Montanaro]

> Eli Zaretskii:

> Etags works by looking at the source at face value, and it doesn't
> expand macros as a C compiler would.  So this:
>
>  Py_LOCAL_INLINE(void) stackdepth_push(basicblock ***sp, basicblock *b, int depth)
>
> looks to its naïve lexical analysis very much like a K&R definition of
> a function and declaration of its arguments:
...
> Patches are welcome, if someone has an idea for how to fix that.

> Franceso:

> Unless some C code wizard steps up and contradicts me, I'd say that it
> is in principle impossible for Etags to detect such macros.  This is a
> work for the --regex feature of Etags, which is thought just for this
> sort of situations.
>
> The Etags man page explains it with examples, as does the info page.

I read the documentation and didn't understand how the regex feature
would help. It seems to be useful to identify other constructs which
you'd like tagged (such as the DEFVAR macro in the Emacs source), not
transform the source in some way. Put another way, I need a
"find/replace" sort of feature, not just a "find" feature.

FWIW, here's the simple shell script I came up with:

#!/bin/bash

# etags gets confused by Python's use of the Py_LOCAL and Py_LOCAL_INLINE
# macros to select compiler-dependent static storage syntax. This script is
# a stand-in for etags which performs the macro expansion manually, then
# pipes the result to etags. $@ are the args to pass to etags.

sed -E -e 's/Py_LOCAL_INLINE *\(([^(]+)\)/static inline \1/' \
    -e 's/Py_LOCAL *\(([^(]+)\)/static \1/' \
    | etags "$@"

I set speedbar-fetch-etags-command to that script. Seems to work well
enough for my needs. It clearly isn't a general solution though. I
suppose I could try to run the source through cpp, though I don't know
if etags is smart enough to use the "# <line> <file>" output of cpp.

I'll take a look at etags.c, but don't hold your breath waiting for a
miracle. :-)

Skip

bug#38511: etags seems to be confused by macros

[Francesco Potortì]

>I read the documentation and didn't understand how the regex feature
>would help. It seems to be useful to identify other constructs which
>you'd like tagged (such as the DEFVAR macro in the Emacs source), not
>transform the source in some way. Put another way, I need a
>"find/replace" sort of feature, not just a "find" feature.

Try this one, and let us know if it satisfies your needs:

etags --regex='/Py_LOCAL_INLINE[ \t]*(.*)\n[^(]+(\([^)]+\)/m'

It assumes that Py_LOCAL_INLINE and its arguments all stay on a single
line, but it can be easily changed to waive this assumption.

-- 
Francesco Potortì (ricercatore)        Voice:  +39.050.621.3058
ISTI - Area della ricerca CNR          Mobile: +39.348.8283.107
via G. Moruzzi 1, I-56124 Pisa         Skype:  wnlabisti
(gate 20, 1st floor, room C71)         Web:    http://fly.isti.cnr.it

bug#38511: etags seems to be confused by macros

[Lars Ingebrigtsen]

Francesco Potortì <pot@gnu.org> writes:

> Unless some C code wizard steps up and contradicts me, I'd say that it
> is in principle impossible for Etags to detect such macros.

Yeah, so I think this is a bit out of the remit for etags.  So I'm
closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no