bug#74861: 31.0.50; etags no longer processing shy groups as expected in --regex options

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 13 Dec 2024 11:06:30 -0500
> From: dvilleneuve.4142@gmail.com
> 
> On a C file (foo.c) with the following line:
> 
> DEFINE_FF(TAG func)
> 
> the etags command used to tag "func" (version 26.1):
> 
> etags --regex='/DEFINE_FF *(\(?:TAG \)? *\([^)]+\)/\1/' \
>   --output ETAGS-test foo.c
> 
> produces
> 
> $ od -c ETAGS-test
> 0000000  \f  \n   f   o   o   .   c   ,   2   8  \n   D   E   F   I   N
> 0000020   E   _   F   F   (   T   A   G       f   u   n   c 177   f   u
> 0000040   n   c 001   1   ,   0  \n
> 
> so "func" between \x7f and \x01.
> 
> With versions 27.2 and later, we get:
> $ od -c ETAGS-test
> 0000000  \f  \n   f   o   o   .   c   ,   0  \n
> 0000012
> 
> so no match, and replacing \1 by \2 in the --regex argument
> (admittedly not well defined since there is a single registered group),
> we get:
> 
> $ od -c ETAGS-test
> 0000000  \f  \n   f   o   o   .   c   ,   3   2  \n   D   E   F   I   N
> 0000020   E   _   F   F   (   T   A   G       f   u   n   c 177   T   A
> 0000040   G       f   u   n   c 001   1   ,   0  \n
> 
> so "TAG func" between \x7f and \x01.
> 
> In both cases the result is different from the one from 26.1,
> which I would think is the expected one.
> 
> Using
> 
> etags --regex='/DEFINE_FF *(\(TAG \)? *\([^)]+\)/\2/' \
>   --output ETAGS-test foo.c
> 
> (that is, without a shy group) works as expected, in 26.1 and later.

I'm guessing this is because Emacs 27 switched to the Gnulib's regex
implementation in etags and other lib-src programs, whereas previous
versions used the Emacs's own regex code (which is still used for
Emacs's own regex search and replacement code).

CC'ing Paul Eggert, in the hope he can tell whether this is expected
or not, or how to fix it.