etags name collision.

etags name collision.

[Ergus]

Hi:

Recently I made the already mentioned package gtags-mode in order to use
it with gnu global. Recently a friend using verilog found that gnu
global can use ctags (Universal Ctags) as a backend for verilog as
universal ctags has a set of plugins very complete and works specially
well with verilog.

But in my system there is an issue, because emacs also creates a ctags
executable with similar function but not as good as Universal ctags in
many sences. The questions are:

1) Why do we have etags+ctags executables so far they do more or less
the same work right?.

2) Why do we create an executable with a name that is used by another
very wheel known program.

3) Could we consider to keep only etags and remove the ctags file in
order to let the users to access. GREP had a similar issue some time ago
with egrep rgrep and similes; and they solved that adding command lines
to grep; why not to do the same?

In general when the users want to use ctags I am pretty sure they refer
to universal or exuberant ctags today... But also such executable create
inconsistencies when using TRAMP and the support for languages like Rust
or modern C++ is not good; so maybe an even more radical approach may be
considered. Some distros like Arch Linux explicitly rename it to solve
the conflict, but if we have etags already, do we really need the other
executable?

Best,
Ergus

Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 14:47:36 +0200
> From: Ergus <spacibba@aol.com>
> 
> 1) Why do we have etags+ctags executables so far they do more or less
> the same work right?.

Because ctags produces tags table in a format understood by more
applications than just Emacs.

> 2) Why do we create an executable with a name that is used by another
> very wheel known program.

It's the other way around: ctags was a very old Unix program, and
Emacs developed a GNU version of that program.  Universal ctags came
much later.  So you should ask them why did they decide to use a name
that was already taken.

> 3) Could we consider to keep only etags and remove the ctags file in
> order to let the users to access.

I don't see why we should remove a program just because someone who
uses an incompatible program by the same name should manage his/her
installations.  A simple solution is for that user to not install the
Emacs version of ctags.

> In general when the users want to use ctags I am pretty sure they refer
> to universal or exuberant ctags today... But also such executable create
> inconsistencies when using TRAMP and the support for languages like Rust
> or modern C++ is not good; so maybe an even more radical approach may be
> considered. Some distros like Arch Linux explicitly rename it to solve
> the conflict, but if we have etags already, do we really need the other
> executable?

Emacs doesn't need ctags, but users might need it for working with
other development tools.

Re: etags name collision.

[Dmitry Gutov]

On 11.04.2022 16:18, Eli Zaretskii wrote:
>> 2) Why do we create an executable with a name that is used by another
>> very wheel known program.
> It's the other way around: ctags was a very old Unix program, and
> Emacs developed a GNU version of that program.  Universal ctags came
> much later.  So you should ask them why did they decide to use a name
> that was already taken.
> 

Even the very old version of ctags that's distributed by most of 
GNU/Linux distributions these days (Ctags 5.9~svn20110310) supports more 
languages than etags. Like ~50% more languages.

So it shouldn't come as a surprise that the our version of ctags was 
never accepted as a true continuation or replacement.

It's not like we're doing a particularly active development of it either.

Re: etags name collision.

[Ergus]

Hi Eli:

On Mon, Apr 11, 2022 at 04:18:43PM +0300, Eli Zaretskii wrote:
>> Date: Mon, 11 Apr 2022 14:47:36 +0200
>> From: Ergus <spacibba@aol.com>
>>
>> 1) Why do we have etags+ctags executables so far they do more or less
>> the same work right?.
>
>Because ctags produces tags table in a format understood by more
>applications than just Emacs.
>
>> 2) Why do we create an executable with a name that is used by another
>> very wheel known program.
>
>It's the other way around: ctags was a very old Unix program, and
>Emacs developed a GNU version of that program.  Universal ctags came
>much later.  So you should ask them why did they decide to use a name
>that was already taken.
>
For whatever reason their version is much better, maintained and with
more active development, support and people. It's free in all the senses
and provided in some GNU/Linux distros by default or available in 100%
of the repositories... so don't fight them, join then that did a good
work stablishing as the standard. (Do one thing and do it right)

>> 3) Could we consider to keep only etags and remove the ctags file in
>> order to let the users to access.
>
>I don't see why we should remove a program just because someone who
>uses an incompatible program by the same name should manage his/her
>installations.  A simple solution is for that user to not install the
>Emacs version of ctags.
>
There is no consistent way to do that in many distros they don't have
emacs users so they declare emacs incompatible with ctags and the user
needs to choose.

There is not even an option to disable the creation of ctags during
emacs configure/build; so it needs to be removed every time.

>> In general when the users want to use ctags I am pretty sure they refer
>> to universal or exuberant ctags today... But also such executable create
>> inconsistencies when using TRAMP and the support for languages like Rust
>> or modern C++ is not good; so maybe an even more radical approach may be
>> considered. Some distros like Arch Linux explicitly rename it to solve
>> the conflict, but if we have etags already, do we really need the other
>> executable?
>
>Emacs doesn't need ctags

So we shouldn't provide ctags by default. It is like if the Linux kernel
provide git and bash because they are useful. We don't have man power to
maintain it and support all the languages and features Exhuberant or
Universal do. But also in many distros ctags is actually provided by
default.

>but users might need it for working with
>other development tools.
>
The users who use ctags they only know about the other versions like
universal or exuberant; most of them don't even know that emacs provides
a ctags version and it only causes troubles.

At least could we condition the creation of ctags with a build/configure
option??
  Best,
Ergus

Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 04:38:38PM +0300, Dmitry Gutov wrote:
>On 11.04.2022 16:18, Eli Zaretskii wrote:
>>>2) Why do we create an executable with a name that is used by another
>>>very wheel known program.
>>It's the other way around: ctags was a very old Unix program, and
>>Emacs developed a GNU version of that program.  Universal ctags came
>>much later.  So you should ask them why did they decide to use a name
>>that was already taken.
>>
>
>Even the very old version of ctags that's distributed by most of 
>GNU/Linux distributions these days (Ctags 5.9~svn20110310) supports 
>more languages than etags. Like ~50% more languages.
>
>So it shouldn't come as a surprise that the our version of ctags was 
>never accepted as a true continuation or replacement.
>
>It's not like we're doing a particularly active development of it either.

Do one thing and do it right...

Re: etags name collision.

[Kaushal Modi]

On Mon, Apr 11, 2022 at 8:48 AM Ergus <spacibba@aol.com> wrote:
>
> Hi:
>
> Recently I made the already mentioned package gtags-mode in order to use
> it with gnu global. Recently a friend using verilog found that gnu
> global can use ctags (Universal Ctags) as a backend for verilog as
> universal ctags has a set of plugins very complete and works specially
> well with verilog.

I also develop in (system)verilog :)

> 3) Could we consider to keep only etags and remove the ctags file in
> order to let the users to access. GREP had a similar issue some time ago
> with egrep rgrep and similes; and they solved that adding command lines
> to grep; why not to do the same?

Emacs configure comes with a --program-transform-name switch. I add
this to my configure command:
--program-transform-name='s/^ctags$/ctags_emacs/' to avoid clash with
the Universal Ctags 'ctags' executable.

Re: etags name collision.

[Andreas Schwab]

On Apr 11 2022, Dmitry Gutov wrote:

> Even the very old version of ctags that's distributed by most of GNU/Linux
> distributions these days (Ctags 5.9~svn20110310) supports more languages
> than etags. Like ~50% more languages.

That's not the original ctags program, though.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 16:38:38 +0300
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: emacs-devel@gnu.org
> 
> On 11.04.2022 16:18, Eli Zaretskii wrote:
> >> 2) Why do we create an executable with a name that is used by another
> >> very wheel known program.
> > It's the other way around: ctags was a very old Unix program, and
> > Emacs developed a GNU version of that program.  Universal ctags came
> > much later.  So you should ask them why did they decide to use a name
> > that was already taken.
> > 
> 
> Even the very old version of ctags that's distributed by most of 
> GNU/Linux distributions these days (Ctags 5.9~svn20110310) supports more 
> languages than etags. Like ~50% more languages.
> 
> So it shouldn't come as a surprise that the our version of ctags was 
> never accepted as a true continuation or replacement.
> 
> It's not like we're doing a particularly active development of it either.

That's okay: people are free to use the other variants of ctags.  It's
the same situation as with movemail.

Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 15:47:49 +0200
> From: Ergus <spacibba@aol.com>
> Cc: emacs-devel@gnu.org
> 
> >Emacs doesn't need ctags
> 
> So we shouldn't provide ctags by default.

I'm opposed to dropping stuff just because we can.  If you want to add
an option not to install ctags if another version is installed
(similarly to what we do with movemail), that's fine.  But please
don't argue for dropping parts of Emacs unconditionally because you
personally find it not useful, that argument will never fly with me.

Re: etags name collision.

[Andreas Schwab]

On Apr 11 2022, Eli Zaretskii wrote:

> It's the other way around: ctags was a very old Unix program, and
> Emacs developed a GNU version of that program.  Universal ctags came
> much later.  So you should ask them why did they decide to use a name
> that was already taken.

They are both implementation of the POSIX utility ctags, with
extensions.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 15:52:59 +0200
> From: Ergus <spacibba@aol.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> Do one thing and do it right...

And we don't??

Re: etags name collision.

[Óscar Fuentes]

Kaushal Modi <kaushal.modi@gmail.com> writes:

>> 3) Could we consider to keep only etags and remove the ctags file in
>> order to let the users to access. GREP had a similar issue some time ago
>> with egrep rgrep and similes; and they solved that adding command lines
>> to grep; why not to do the same?
>
> Emacs configure comes with a --program-transform-name switch. I add
> this to my configure command:
> --program-transform-name='s/^ctags$/ctags_emacs/' to avoid clash with
> the Universal Ctags 'ctags' executable.

Nifty.

In MSYS2 we simply rm bin/ctags.exe (and the corresponding man file) and
provide universal-ctags as a dependency. Arch Linux renames (with `mv')
"ctags" to "ctags.emacs".

Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 05:09:53PM +0300, Eli Zaretskii wrote:
>> Date: Mon, 11 Apr 2022 15:47:49 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: emacs-devel@gnu.org
>>
>> >Emacs doesn't need ctags
>>
>> So we shouldn't provide ctags by default.
>
>I'm opposed to dropping stuff just because we can.  If you want to add
>an option not to install ctags if another version is installed
>(similarly to what we do with movemail), that's fine.

That may be an acceptable solution... But I will need to learn autotools
to implement that myself...

>But please
>don't argue for dropping parts of Emacs unconditionally because you
>personally find it not useful, that argument will never fly with me.

I know Eli; But at some point you find that situations like this create
more confusion than benefit to the users and require some developer
support (man power) we lack of. I usually have the same feeling in the
opposite direction; when adding features that are not popular enough to
be maintained in the long therm... but that's another discussion.

Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 05:11:22PM +0300, Eli Zaretskii wrote:
>> Date: Mon, 11 Apr 2022 15:52:59 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
>>
>> Do one thing and do it right...
>
>And we don't??
>
ctags is a different "thing" than the emacs "thing" and with the current
support it has is not right (or good enough)... so... no one thing and
not right either...

[PATCH] Re: etags name collision.

[Ergus]

[-- Attachment #1: Type: text/plain, Size: 471 bytes --]

On Mon, Apr 11, 2022 at 05:09:53PM +0300, Eli Zaretskii wrote:
>> Date: Mon, 11 Apr 2022 15:47:49 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: emacs-devel@gnu.org
>>
>> >Emacs doesn't need ctags
>>
>> So we shouldn't provide ctags by default.
>
>I'm opposed to dropping stuff just because we can.  If you want to add
>an option not to install ctags if another version is installed
>(similarly to what we do with movemail), that's fine.

What about the attached patch?


[-- Attachment #2: ctags.patch --]
[-- Type: text/plain, Size: 1595 bytes --]

diff --git a/configure.ac b/configure.ac
index 185e4d0862..25e45d9ac6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -267,6 +267,19 @@ AC_DEFUN
 fi
 AC_SUBST([with_mailutils])
 
+AC_ARG_WITH([ctags],
+  [AS_HELP_STRING([--with-ctags],
+     [rely on System ctags; this is the default if Universal ctags or
+      Exuberant ctags is installed])],
+  [],
+  [with_ctags=$with_features
+   if test "$with_ctags" = yes; then
+     (ctags --version) >/dev/null 2>&1 || with_ctags=no
+   fi])
+if test "$with_ctags" = no; then
+  with_ctags=
+fi
+
 AC_ARG_WITH([pop],
   [AS_HELP_STRING([--with-pop],
      [Support POP mail retrieval if Emacs movemail is used (not recommended,
diff --git a/lib-src/Makefile.in b/lib-src/Makefile.in
index 0453b93506..ee17576686 100644
--- a/lib-src/Makefile.in
+++ b/lib-src/Makefile.in
@@ -84,6 +84,9 @@ libexecdir=
 # Nonempty if Emacs can assume Mailutils is installed.
 with_mailutils=@with_mailutils@
 
+# Nonempty if Emacs can assume ctags is installed.
+with_ctags=@with_ctags@
+
 # Directory for local state files for all programs.
 localstatedir=@localstatedir@
 
@@ -144,8 +147,8 @@ HAIKU_CFLAGS=
 CLIENTW = @CLIENTW@
 
 # Things that a user might actually run, which should be installed in bindir.
-INSTALLABLES = etags${EXEEXT} ctags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) \
-               ebrowse${EXEEXT}
+INSTALLABLES = etags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) ebrowse${EXEEXT} \
+	       $(if $(with_ctags), , ctags${EXEEXT})
 
 # Things that Emacs runs internally, or during the build process,
 #  which should not be installed in bindir.

Re: [PATCH] Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 17:46:35 +0200
> From: Ergus <spacibba@aol.com>
> Cc: emacs-devel@gnu.org
> 
> >I'm opposed to dropping stuff just because we can.  If you want to add
> >an option not to install ctags if another version is installed
> >(similarly to what we do with movemail), that's fine.
> 
> What about the attached patch?

The code should test whether another version of 'ctags' is already
installed, and if so, that it isn't our 'ctags'.  _Then_ we could
default to not installing our 'ctags'.  Your proposal doesn't make
that test, so please add it.

Thanks.

Re: [PATCH] Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 06:51:58PM +0300, Eli Zaretskii wrote:
>> Date: Mon, 11 Apr 2022 17:46:35 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: emacs-devel@gnu.org
>>
>> >I'm opposed to dropping stuff just because we can.  If you want to add
>> >an option not to install ctags if another version is installed
>> >(similarly to what we do with movemail), that's fine.
>>
>> What about the attached patch?
>
>The code should test whether another version of 'ctags' is already
>installed, and if so, that it isn't our 'ctags'.  _Then_ we could
>default to not installing our 'ctags'.  Your proposal doesn't make
>that test, so please add it.
>
>Thanks.
>
I do this tests like with mailutils and movemail:

(ctags --version) >/dev/null 2>&1 || with_ctags=no

Isn't this enough?

Re: [PATCH] Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 18:19:42 +0200
> From: Ergus <spacibba@aol.com>
> Cc: emacs-devel@gnu.org
> 
> On Mon, Apr 11, 2022 at 06:51:58PM +0300, Eli Zaretskii wrote:
> 
> >The code should test whether another version of 'ctags' is already
> >installed, and if so, that it isn't our 'ctags'.  _Then_ we could
> >default to not installing our 'ctags'.  Your proposal doesn't make
> >that test, so please add it.
> >
> >Thanks.
> >
> I do this tests like with mailutils and movemail:
> 
> (ctags --version) >/dev/null 2>&1 || with_ctags=no
> 
> Isn't this enough?

No, because AFAIU this test will succeed also if the installed ctags
is (an older version) of the program that came with (an older version)
of Emacs.  You need to make sure the text emitted by --version does
NOT include "GNU Emacs".

Re: etags name collision.

[Stefan Monnier]

Eli Zaretskii [2022-04-11 17:09:53] wrote:
> I'm opposed to dropping stuff just because we can.  If you want to add
> an option not to install ctags if another version is installed
> (similarly to what we do with movemail), that's fine.  But please
> don't argue for dropping parts of Emacs unconditionally because you
> personally find it not useful, that argument will never fly with me.

Maybe we should install it under a different name, to avoid
the conflict but without losing the functionality?


        Stefan

Re: etags name collision.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Ergus <spacibba@aol.com>,  emacs-devel@gnu.org
> Date: Mon, 11 Apr 2022 12:46:03 -0400
> 
> Eli Zaretskii [2022-04-11 17:09:53] wrote:
> > I'm opposed to dropping stuff just because we can.  If you want to add
> > an option not to install ctags if another version is installed
> > (similarly to what we do with movemail), that's fine.  But please
> > don't argue for dropping parts of Emacs unconditionally because you
> > personally find it not useful, that argument will never fly with me.
> 
> Maybe we should install it under a different name, to avoid
> the conflict but without losing the functionality?

No, that would be confusing and will break those who do still use our
ctags.

I think detecting that another kind of ctags is installed is the best
and safest way to have the cake and eat it, too.

Re: etags name collision.

[Stefan Monnier]

>> Maybe we should install it under a different name, to avoid
>> the conflict but without losing the functionality?
> No, that would be confusing

If we install it under a name like `firefox`?  Yes, no doubt.
But I'm sure we can come up with a sane enough name to avoid this problem.

> and will break those who do still use our ctags.

Yes, that's the main downside.
But it'd be a small matter of adding an alias or symlink, so it's not
that bad.

> I think detecting that another kind of ctags is installed is the best
> and safest way to have the cake and eat it, too.

Rather than "install vs not-install" the test could decide "install as
`ctags` vs install as `somethingelse`", which is closer to having our
cake and eating it too since the user can then have both installed at
the same time.

Still, an install-time test has the downside that it won't adapt if
`ctags` is installed after Emacs is built/installed, as is probably the
case for most users nowadays (who don't build Emacs themselves but
install from a distro or from our prebuilt tarballs).

AFAICT it won't make a big difference for GNU/Linux distros since these
already go through the trouble of renaming our `ctags` (it might just
make their job a bit easier and avoid inconsistencies between distros
where every distro may choose a slightly different name).

BTW, regarding the state of our `ctags` compared to the other ones.
A few years back I remember someone comparing them and finding out that
we support fewer languages but that for some of those languages we
provided better results.  If that's not the case any more, maybe we
should check to see if we could make `etags.el` use "Exhuberant ctags"
when available so as to benefit from the better tool.


        Stefan

Re: etags name collision.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: spacibba@aol.com,  emacs-devel@gnu.org
> Date: Mon, 11 Apr 2022 13:41:00 -0400
> 
> > I think detecting that another kind of ctags is installed is the best
> > and safest way to have the cake and eat it, too.
> 
> Rather than "install vs not-install" the test could decide "install as
> `ctags` vs install as `somethingelse`", which is closer to having our
> cake and eating it too since the user can then have both installed at
> the same time.

No, the intent is exactly to let users have the program they want
under the name "ctags".  Everything else, including renaming, is left
to the users themsleves, because we have no way of second-guessing
their habits.  And as someone else pointed out, we already support
installing under a different name, if the user knows what is that
name.

> Still, an install-time test has the downside that it won't adapt if
> `ctags` is installed after Emacs is built/installed, as is probably the
> case for most users nowadays (who don't build Emacs themselves but
> install from a distro or from our prebuilt tarballs).

I don't think we need to make too much out of it.  the problem is
easily solved manually, so we just make it a tad easier in one simple
use case.

> BTW, regarding the state of our `ctags` compared to the other ones.
> A few years back I remember someone comparing them and finding out that
> we support fewer languages but that for some of those languages we
> provided better results.  If that's not the case any more, maybe we
> should check to see if we could make `etags.el` use "Exhuberant ctags"
> when available so as to benefit from the better tool.

That, too, is already covered by the installation of Exhuberant and
other variants.

Re: etags name collision.

[Ergus]

Hi Stefan:

I totally agree with you in everything here and this was actually the
reason to start this thread.

There are two points here:

1) We are talking about the ctags executable, not about the etags
one... emacs does not use such ctags executable at all and it is
compiled and installed unconditionally.

2) About etags IMHO, if we could support "Exhuberant/Universal ctags"
(even if not enabled by default) is a much better solution in the long
path and an important step because it will save us to maintain and
support all the languages that they already maintain but also in a
better way.  Even no so weird languages like C++ or some very popular
ones like Rust have very poor or null support in etags, the database
creation needs to filter files manually and it is sadly useless for
tramp users that frequently have ctags in remote hosts but not etags or
emacs.

>
>BTW, regarding the state of our `ctags` compared to the other ones.
>A few years back I remember someone comparing them and finding out that
>we support fewer languages but that for some of those languages we
>provided better results.  If that's not the case any more, maybe we
>should check to see if we could make `etags.el` use "Exhuberant ctags"
>when available so as to benefit from the better tool.
>
I am not sure when such comparison was made but at the moment Universal
Ctags is very powerful and modular and superior Compared with
"Exhuberant ctags".  It works pretty well as backend for other programs
like gtags; adding external modules of backends is relatively easy, so
it is in constant development and active... You just need to give a look
to their repository:

https://github.com/universal-ctags/ctags

>
>        Stefan
>
vim users get this working out of the box for all the languages while we
have very poor support for it. 

Re: etags name collision.

[Ulrich Mueller]

>>>>> On Mon, 11 Apr 2022, Eli Zaretskii wrote:

>> Rather than "install vs not-install" the test could decide "install as
>> `ctags` vs install as `somethingelse`", which is closer to having our
>> cake and eating it too since the user can then have both installed at
>> the same time.

> No, the intent is exactly to let users have the program they want
> under the name "ctags".  Everything else, including renaming, is left
> to the users themsleves, because we have no way of second-guessing
> their habits.  And as someone else pointed out, we already support
> installing under a different name, if the user knows what is that
> name.

+1

There already are autoconf's --program-prefix, --program-suffix and
--program-transform-name options which can do everything that is needed.

For example, Gentoo uses these configure options to install Emacs' ctags
as (e.g.) "ctags-emacs-28". Exuberant/universal ctags is installed as
"exuberant-ctags". "ctags" is a symlink to one of them and can be
updated by the user.

>> Still, an install-time test has the downside that it won't adapt if
>> `ctags` is installed after Emacs is built/installed, as is probably the
>> case for most users nowadays (who don't build Emacs themselves but
>> install from a distro or from our prebuilt tarballs).

I tend to agree. From a distro point of view, testing for an installed
ctags binary (with the purpose to avoid file name collisions) is not
helpful. The set of files in the installed image should not depend on
random factors like package installation order.

> I don't think we need to make too much out of it.  the problem is
> easily solved manually, so we just make it a tad easier in one simple
> use case.

Re: [PATCH] Re: etags name collision.

[Ergus]

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

On Mon, Apr 11, 2022 at 07:40:29PM +0300, Eli Zaretskii wrote:
>> Date: Mon, 11 Apr 2022 18:19:42 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: emacs-devel@gnu.org
>>
>> On Mon, Apr 11, 2022 at 06:51:58PM +0300, Eli Zaretskii wrote:
>>
>> >The code should test whether another version of 'ctags' is already
>> >installed, and if so, that it isn't our 'ctags'.  _Then_ we could
>> >default to not installing our 'ctags'.  Your proposal doesn't make
>> >that test, so please add it.
>> >
>> >Thanks.
>> >
>> I do this tests like with mailutils and movemail:
>>
>> (ctags --version) >/dev/null 2>&1 || with_ctags=no
>>
>> Isn't this enough?
>
>No, because AFAIU this test will succeed also if the installed ctags
>is (an older version) of the program that came with (an older version)
>of Emacs.  You need to make sure the text emitted by --version does
>NOT include "GNU Emacs".
>
Now?


[-- Attachment #2: ctags.patch --]
[-- Type: text/plain, Size: 1610 bytes --]

diff --git a/configure.ac b/configure.ac
index 185e4d0862..ace80aed56 100644
--- a/configure.ac
+++ b/configure.ac
@@ -267,6 +267,19 @@ AC_DEFUN
 fi
 AC_SUBST([with_mailutils])
 
+AC_ARG_WITH([ctags],
+  [AS_HELP_STRING([--with-ctags],
+     [rely on System ctags; this is the default if Universal ctags or
+      Exuberant ctags is installed])],
+  [],
+  [with_ctags=$with_features
+   if test "$with_ctags" = yes; then
+     (ctags --version | grep "GNU Emacs") 2>/dev/null || with_ctags=no
+   fi])
+if test "$with_ctags" = no; then
+  with_ctags=
+fi
+
 AC_ARG_WITH([pop],
   [AS_HELP_STRING([--with-pop],
      [Support POP mail retrieval if Emacs movemail is used (not recommended,
diff --git a/lib-src/Makefile.in b/lib-src/Makefile.in
index 0453b93506..ee17576686 100644
--- a/lib-src/Makefile.in
+++ b/lib-src/Makefile.in
@@ -84,6 +84,9 @@ libexecdir=
 # Nonempty if Emacs can assume Mailutils is installed.
 with_mailutils=@with_mailutils@
 
+# Nonempty if Emacs can assume ctags is installed.
+with_ctags=@with_ctags@
+
 # Directory for local state files for all programs.
 localstatedir=@localstatedir@
 
@@ -144,8 +147,8 @@ HAIKU_CFLAGS=
 CLIENTW = @CLIENTW@
 
 # Things that a user might actually run, which should be installed in bindir.
-INSTALLABLES = etags${EXEEXT} ctags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) \
-               ebrowse${EXEEXT}
+INSTALLABLES = etags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) ebrowse${EXEEXT} \
+	       $(if $(with_ctags), , ctags${EXEEXT})
 
 # Things that Emacs runs internally, or during the build process,
 #  which should not be installed in bindir.

Re: [PATCH] Re: etags name collision.

[Ulrich Mueller]

>>>>> On Mon, 11 Apr 2022, Ergus  wrote:

> Now?

> diff --git a/configure.ac b/configure.ac
> index 185e4d0862..ace80aed56 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -267,6 +267,19 @@ AC_DEFUN
>  fi
>  AC_SUBST([with_mailutils])
 
> +AC_ARG_WITH([ctags],
> +  [AS_HELP_STRING([--with-ctags],
> +     [rely on System ctags; this is the default if Universal ctags or
> +      Exuberant ctags is installed])],
> +  [],
> +  [with_ctags=$with_features
> +   if test "$with_ctags" = yes; then
> +     (ctags --version | grep "GNU Emacs") 2>/dev/null || with_ctags=no

Shouldn't this use the actual name under which Emacs will install ctags?
That is, respect AC_ARG_PROGRAM?

In its current form, it would break installation of the Gentoo package
(or require us adding an explicit --without-ctags).

> +   fi])
> +if test "$with_ctags" = no; then
> +  with_ctags=
> +fi

I still think that any test for an installed binary is a bad idea, from
a distro point of view. Note that distros typically build packages in an
environment that is different from the one of the final target system.

Re: [PATCH] Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 09:39:36PM +0200, Ulrich Mueller wrote:
>>>>>> On Mon, 11 Apr 2022, Ergus  wrote:
>
>> Now?
>
>> diff --git a/configure.ac b/configure.ac
>> index 185e4d0862..ace80aed56 100644
>> --- a/configure.ac
>> +++ b/configure.ac
>> @@ -267,6 +267,19 @@ AC_DEFUN
>>  fi
>>  AC_SUBST([with_mailutils])
>
>> +AC_ARG_WITH([ctags],
>> +  [AS_HELP_STRING([--with-ctags],
>> +     [rely on System ctags; this is the default if Universal ctags or
>> +      Exuberant ctags is installed])],
>> +  [],
>> +  [with_ctags=$with_features
>> +   if test "$with_ctags" = yes; then
>> +     (ctags --version | grep "GNU Emacs") 2>/dev/null || with_ctags=no
>
>Shouldn't this use the actual name under which Emacs will install ctags?
>That is, respect AC_ARG_PROGRAM?
>
Not needed, if it uses a different name there is no collision, so the
test is not needed.

>In its current form, it would break installation of the Gentoo package
>(or require us adding an explicit --without-ctags).
>
The option --with-ctags means rely on System ctags. I don't think you
need anything new.

>> +   fi])
>> +if test "$with_ctags" = no; then
>> +  with_ctags=
>> +fi
>
>I still think that any test for an installed binary is a bad idea, from
>a distro point of view. Note that distros typically build packages in an
>environment that is different from the one of the final target system.
>
Here I agree

Re: [PATCH] Re: etags name collision.

[Ulrich Mueller]

>>>>> On Mon, 11 Apr 2022, Ergus  wrote:

>>> +AC_ARG_WITH([ctags],
>>> +  [AS_HELP_STRING([--with-ctags],
>>> +     [rely on System ctags; this is the default if Universal ctags or
>>> +      Exuberant ctags is installed])],
>>> +  [],
>>> +  [with_ctags=$with_features
>>> +   if test "$with_ctags" = yes; then
>>> +     (ctags --version | grep "GNU Emacs") 2>/dev/null || with_ctags=no
>> 
>> Shouldn't this use the actual name under which Emacs will install ctags?
>> That is, respect AC_ARG_PROGRAM?
>> 
> Not needed, if it uses a different name there is no collision, so the
> test is not needed.

If universal ctags is installed as "ctags", above test would detect that
the installed version is not Emacs ctags. So, installation of ctags by
Emacs would be suppressed, even if it was under a different name.

Or am I missing something here?

> The option --with-ctags means rely on System ctags. I don't think you
> need anything new.

Universal ctags isn't more "system" than Emacs. They're both packages.
(Again, this is from a distro point of view.)

Re: [PATCH] Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 11:04:56PM +0200, Ulrich Mueller wrote:
>>>>>> On Mon, 11 Apr 2022, Ergus  wrote:
>
>>>> +AC_ARG_WITH([ctags],
>>>> +  [AS_HELP_STRING([--with-ctags],
>>>> +     [rely on System ctags; this is the default if Universal ctags or
>>>> +      Exuberant ctags is installed])],
>>>> +  [],
>>>> +  [with_ctags=$with_features
>>>> +   if test "$with_ctags" = yes; then
>>>> +     (ctags --version | grep "GNU Emacs") 2>/dev/null || with_ctags=no
>>>
>>> Shouldn't this use the actual name under which Emacs will install ctags?
>>> That is, respect AC_ARG_PROGRAM?
>>>
>> Not needed, if it uses a different name there is no collision, so the
>> test is not needed.
>
>If universal ctags is installed as "ctags", above test would detect that
>the installed version is not Emacs ctags. So, installation of ctags by
>Emacs would be suppressed, even if it was under a different name.
>
>Or am I missing something here?
>
But that's exactly the idea... to do what we do with movemail...

>> The option --with-ctags means rely on System ctags. I don't think you
>> need anything new.
>
>Universal ctags isn't more "system" than Emacs. They're both packages.
>(Again, this is from a distro point of view.)
>
emacs's ctags is not a package but just an executable inside another
package.

Re: etags name collision.

[Ergus]

On Mon, Apr 11, 2022 at 01:41:00PM -0400, Stefan Monnier wrote:
>BTW, regarding the state of our `ctags` compared to the other ones.
>A few years back I remember someone comparing them and finding out that
>we support fewer languages but that for some of those languages we
>provided better results.  If that's not the case any more, maybe we
>should check to see if we could make `etags.el` use "Exhuberant ctags"
>when available so as to benefit from the better tool.
>
>
>        Stefan
>
Another unmentioned advantage of universal ctags is the supported output
formats. One of them is json, which may simplify and improve the parsing
using our new json API...

Re: [PATCH] Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 21:53:50 +0200
> From: Ergus <spacibba@aol.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> >> +   fi])
> >> +if test "$with_ctags" = no; then
> >> +  with_ctags=
> >> +fi
> >
> >I still think that any test for an installed binary is a bad idea, from
> >a distro point of view. Note that distros typically build packages in an
> >environment that is different from the one of the final target system.
> >
> Here I agree

How else to test whether this is needed?  I'm okay with having
"--without-ctags" with no test, but then the default will have to be
to install our ctags.  With the test, we could refrain from installing
it if the test says so.

I'm also okay with leaving things as they are now, obviously, if this
change brings more problems than it solves.  I don't consider the
current situation bad enough to necessitate any changes.

Re: [PATCH] Re: etags name collision.

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> I'm also okay with leaving things as they are now, obviously, if this
> change brings more problems than it solves.  I don't consider the
> current situation bad enough to necessitate any changes.

+1.  I've always been using the Emacs-installed ctags binary to generate
ctags tables without any problems.

Re: etags name collision.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 1) Why do we have etags+ctags executables so far they do more or less
  > the same work right?.

What else would we do?

  > GREP had a similar issue some time ago
  > with egrep rgrep and similes; and they solved that adding command lines
  > to grep; why not to do the same?

What does "adding command lines to grep" mean, concretely?
It isn't clear to me.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH] Re: etags name collision.

[Ulrich Mueller]

>>>>> On Tue, 12 Apr 2022, Eli Zaretskii wrote:

>> Date: Mon, 11 Apr 2022 21:53:50 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
>> 
>> >I still think that any test for an installed binary is a bad idea, from
>> >a distro point of view. Note that distros typically build packages in an
>> >environment that is different from the one of the final target system.
>> >
>> Here I agree

> How else to test whether this is needed?  I'm okay with having
> "--without-ctags" with no test, but then the default will have to be
> to install our ctags.

That sounds good.

> With the test, we could refrain from installing it if the test says
> so.

But then the test should be more specific, and check if there would be a
file collision at the actual target location (with the name modified by
--program-transform-name, if applicable). If there's no collision (e.g.
Emacs ctags has a different name) then Emacs should install it.

> I'm also okay with leaving things as they are now, obviously, if this
> change brings more problems than it solves.  I don't consider the
> current situation bad enough to necessitate any changes.

Things are like this since more than a decade, and obviously distros can
cope with the status quo.

Re: [PATCH] Re: etags name collision.

[Alfred M. Szmidt]

   > >> +   fi])
   > >> +if test "$with_ctags" = no; then
   > >> +  with_ctags=
   > >> +fi
   > >
   > >I still think that any test for an installed binary is a bad idea, from
   > >a distro point of view. Note that distros typically build packages in an
   > >environment that is different from the one of the final target system.
   > >
   > Here I agree

   How else to test whether this is needed?  I'm okay with having
   "--without-ctags" with no test, but then the default will have to be
   to install our ctags.  With the test, we could refrain from installing
   it if the test says so.

I think that is a good idea.  The ones suggesting to "DWIM" magic are
prone for error and confusion, if you _really_ need to install ctags
(or etags) as something else, ./configure already supports the means
for that.

Re: etags name collision.

[Alfred M. Szmidt]

     > GREP had a similar issue some time ago
     > with egrep rgrep and similes; and they solved that adding command lines
     > to grep; why not to do the same?

   What does "adding command lines to grep" mean, concretely?
   It isn't clear to me.

I think what is being refered to is that the commands egrep/fgrep are
considered deprecated infavor of using "grep -E" and "grep -F"
directly.

GNU grep did not add those command line options to solve "similar
issue" though.  Why they added it is of a entierly different nature,
namley POSIX requires it and most Unix grep have it.

Re: etags name collision.

[Alfred M. Szmidt]

   >> Do one thing and do it right...
   >
   >And we don't??

   ctags is a different "thing" than the emacs "thing" and with the current
   support it has is not right (or good enough)... so... no one thing and
   not right either...

Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
ctags is compatible with the Emacs ctags output (which is why it
exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
mg(1) on those systems, and their output is at odds with what is
standardized by POSIX.

So really, you're suggesting to remove a standardized utility and
replace it with non-standard ones that produce incompatible output
from what is generally accepted.

Re: [PATCH] Re: etags name collision.

[Alfred M. Szmidt]

   >> The option --with-ctags means rely on System ctags. I don't think you
   >> need anything new.
   >
   >Universal ctags isn't more "system" than Emacs. They're both packages.
   >(Again, this is from a distro point of view.)

   emacs's ctags is not a package but just an executable inside another
   package.

On some systems /usr/bin/ctags is always GNU Emacs ctags (e.g., Gentoo
GNU/Linux); not Exuberant Ctags which is installed as exuberant-ctags.

If Emacs Ctags is missing something, I'm sure Eli et al would be very
happy to see such patches.

Re: [PATCH] Re: etags name collision.

[Ulrich Mueller]

>>>>> On Tue, 12 Apr 2022, Alfred M Szmidt wrote:

> On some systems /usr/bin/ctags is always GNU Emacs ctags (e.g., Gentoo
> GNU/Linux); not Exuberant Ctags which is installed as exuberant-ctags.

The last part is correct. However, the ctags flavour can be configured:

# ls -l /usr/bin/ctags
lrwxrwxrwx 1 root root 14 Apr  5 09:31 /usr/bin/ctags -> ctags-emacs-28
# eselect ctags list
Available ctags symlink targets:
  [1]   ctags-emacs-28 *
  [2]   exuberant-ctags
# eselect ctags set 2
Switching ctags to exuberant-ctags ...
# ls -l /usr/bin/ctags
lrwxrwxrwx 1 root root 15 Apr 12 09:32 /usr/bin/ctags -> exuberant-ctags

Re: etags name collision.

[Thierry Volpiatto]

Eli Zaretskii <eliz@gnu.org> writes:

> And as someone else pointed out, we already support installing under a
> different name, if the user knows what is that name.

Alternatively, I am installing my different emacs versions with

make install bindir=/usr/local/sbin/emacs-<version> infodir=/usr/local/share/info-<version>

and then I symlink the contents of sbin/emacs-<version>/ to
/usr/local/bin. Just ommiting symlinking sbin/emacs-<version>/ctags may
fix this.

-- 
Thierry

Re: etags name collision.

[Andreas Schwab]

On Apr 12 2022, Alfred M. Szmidt wrote:

> Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
> ctags is compatible with the Emacs ctags output (which is why it
> exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
> mg(1) on those systems, and their output is at odds with what is
> standardized by POSIX.

Exuberant ctags was originally distributed with vim, so it is tailored
for that.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 03:16:50AM -0400, Alfred M. Szmidt wrote:
>   >> Do one thing and do it right...
>   >
>   >And we don't??
>
>   ctags is a different "thing" than the emacs "thing" and with the current
>   support it has is not right (or good enough)... so... no one thing and
>   not right either...
>
>Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
>ctags is compatible with the Emacs ctags output (which is why it
>exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
>mg(1) on those systems, and their output is at odds with what is
>standardized by POSIX.
>
In what sense it is not standarized by POSIX?

I use OpenBSD in my servers daily and all my team around use vim and
neovim with Universal Ctags with no issue.  Universal ctags can generate
emacs tags, json, legacy and modern ctags, xref... if there is some
problem the right to do is to open an issue in their project...

Many distros are already renaming our file and nobody notice.

>So really, you're suggesting to remove a standardized utility and
>replace it with non-standard ones that produce incompatible output
>from what is generally accepted.
>
I am suggesting to avoid the forced installation of a utility that we
are not maintaining very well for another very well maintained, with
more languages, support and formats.

Re: etags name collision.

[Po Lu]

Ergus <spacibba@aol.com> writes:

> I use OpenBSD in my servers daily and all my team around use vim and
> neovim with Universal Ctags with no issue.  Universal ctags can generate
> emacs tags, json, legacy and modern ctags, xref... if there is some
> problem the right to do is to open an issue in their project...

vim and neovim are not vi and mg, and are not standardized utilities
either.

Re: [PATCH] Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 03:21:48AM -0400, Alfred M. Szmidt wrote:
>   >> The option --with-ctags means rely on System ctags. I don't think you
>   >> need anything new.
>   >
>   >Universal ctags isn't more "system" than Emacs. They're both packages.
>   >(Again, this is from a distro point of view.)
>
>   emacs's ctags is not a package but just an executable inside another
>   package.
>
>On some systems /usr/bin/ctags is always GNU Emacs ctags (e.g., Gentoo
>GNU/Linux); not Exuberant Ctags which is installed as exuberant-ctags.
>
>If Emacs Ctags is missing something, I'm sure Eli et al would be very
>happy to see such patches.
>
Emacs Ctags IS sadly missing a lot... from Modern C++ to Rust,
Javascript family and so on. I don't think a single man or the small
emacs team can compete with universal-ctags at the moment in any sense:
modular design, extensibility, output formats, modern parsing
infrastructure, filters... etc.

Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 06:51:11PM +0800, Po Lu wrote:
>Ergus <spacibba@aol.com> writes:
>
>> I use OpenBSD in my servers daily and all my team around use vim and
>> neovim with Universal Ctags with no issue.  Universal ctags can generate
>> emacs tags, json, legacy and modern ctags, xref... if there is some
>> problem the right to do is to open an issue in their project...
>
>vim and neovim are not vi and mg, and are not standardized utilities
>either.
>
So you mean that we should maintain this executable with an old name
that collides with a better modern one; useful for all the vim (and
other editors) users, just to support two editors that are not emacs,
very popular and with minimal maintenance...

Any way I am tired of this pointless discussion... When things start
like this in the list we end up going nowhere.

Best,
Ergus

Re: [PATCH] Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 07:03:23AM +0200, Ulrich Mueller wrote:
>>>>>> On Tue, 12 Apr 2022, Eli Zaretskii wrote:
>
>>> Date: Mon, 11 Apr 2022 21:53:50 +0200
>>> From: Ergus <spacibba@aol.com>
>>> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
>>>
>>> >I still think that any test for an installed binary is a bad idea, from
>>> >a distro point of view. Note that distros typically build packages in an
>>> >environment that is different from the one of the final target system.
>>> >
>>> Here I agree
>
>> How else to test whether this is needed?  I'm okay with having
>> "--without-ctags" with no test, but then the default will have to be
>> to install our ctags.
>
>That sounds good.
>
>> With the test, we could refrain from installing it if the test says
>> so.
>
>But then the test should be more specific, and check if there would be a
>file collision at the actual target location (with the name modified by
>--program-transform-name, if applicable). If there's no collision (e.g.
>Emacs ctags has a different name) then Emacs should install it.
>
If we don't do that with mailutils why should we do it with ctags... It
will be more complex and error prone. I would be even simpler and just
add the option --without-ctags to not build our ctags on demand
unconditionally, keeping things as they are now by default.

as a plus we could make: --with-ctags=no, to not build; and we could
allow to do things like: --with-ctags=ctags.emacs to explicitly create
it as ctags.emacs... But I am sure such method will break some standard
or development agreement.


>> I'm also okay with leaving things as they are now, obviously, if this
>> change brings more problems than it solves.  I don't consider the
>> current situation bad enough to necessitate any changes.
>
>Things are like this since more than a decade, and obviously distros can
>cope with the status quo.

1) Then exuberant ctags was not so popular
2) The popular languages then were very different than today (even C++
was very different then)
3) There was not universal ctags which supports all of them and our formats.

Re: etags name collision.

[Dmitry Gutov]

On 12.04.2022 13:51, Po Lu wrote:
> Ergus<spacibba@aol.com>  writes:
> 
>> I use OpenBSD in my servers daily and all my team around use vim and
>> neovim with Universal Ctags with no issue.  Universal ctags can generate
>> emacs tags, json, legacy and modern ctags, xref... if there is some
>> problem the right to do is to open an issue in their project...
> vim and neovim are not vi and mg, and are not standardized utilities
> either.

Are you saying vi or mg, as opposed to vim and neovim, cannot work with 
the tags output by universal ctags?

Re: etags name collision.

[Po Lu]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Are you saying vi or mg, as opposed to vim and neovim, cannot work
> with the tags output by universal ctags?

I'm not, but Alfred evidently is.

Re: [PATCH] Re: etags name collision.

[Eli Zaretskii]

> Date: Tue, 12 Apr 2022 13:13:02 +0200
> From: Ergus <spacibba@aol.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> >But then the test should be more specific, and check if there would be a
> >file collision at the actual target location (with the name modified by
> >--program-transform-name, if applicable). If there's no collision (e.g.
> >Emacs ctags has a different name) then Emacs should install it.
> >
> If we don't do that with mailutils why should we do it with ctags...

Our mailutils is a security risk of sorts, because it doesn't support
modern secured protocols for fetching email.

The ctags issue is different: there's no security risks involved, only
functionality and features.  Thus, it's justified to treat these cases
differently, as regards to the default.

Re: [PATCH] Re: etags name collision.

[Eli Zaretskii]

> Date: Tue, 12 Apr 2022 14:48:34 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: ulm@gentoo.org, emacs-devel@gnu.org
> 
> Our mailutils is a security risk of sorts, 
  ^^^^^^^^^^^^^
I meant "our movemail".  Sorry.

Re: [PATCH] Re: etags name collision.

[Eli Zaretskii]

> Date: Mon, 11 Apr 2022 21:19:33 +0200
> From: Ergus <spacibba@aol.com>
> Cc: emacs-devel@gnu.org
> 
> >> (ctags --version) >/dev/null 2>&1 || with_ctags=no
> >>
> >> Isn't this enough?
> >
> >No, because AFAIU this test will succeed also if the installed ctags
> >is (an older version) of the program that came with (an older version)
> >of Emacs.  You need to make sure the text emitted by --version does
> >NOT include "GNU Emacs".
> >
> Now?

Seems like this idea makes some people unhappy, but anyway:

> +     (ctags --version | grep "GNU Emacs") 2>/dev/null || with_ctags=no

I think you want

   (ctags --version | grep "GNU Emacs") >/dev/null 2>&1 || with_ctags=no

here.

> -INSTALLABLES = etags${EXEEXT} ctags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) \
> -               ebrowse${EXEEXT}
> +INSTALLABLES = etags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) ebrowse${EXEEXT} \
> +	       $(if $(with_ctags), , ctags${EXEEXT})

Isn't it easier to replace the literal "ctags" with a value computed
by the configure script, than have configure compute a flag variable,
and then use $(if ...) in the Makefile?

Re: [PATCH] Re: etags name collision.

[Ulrich Mueller]

>>>>> On Tue, 12 Apr 2022, Ergus  wrote:

>> But then the test should be more specific, and check if there would be a
>> file collision at the actual target location (with the name modified by
>> --program-transform-name, if applicable). If there's no collision (e.g.
>> Emacs ctags has a different name) then Emacs should install it.

> If we don't do that with mailutils why should we do it with ctags...
> It will be more complex and error prone.

Because it's a different problem. The mailutils flag is there because of
security concerns with the bundled movemail. There is no file collision
in that case, because Emacs doesn't install its movemail program in
/usr/bin but in /usr/libexec/emacs/.

> I would be even simpler and just add the option --without-ctags to not
> build our ctags on demand unconditionally, keeping things as they are
> now by default.

Right. The test doesn't add much value (if any).

> as a plus we could make: --with-ctags=no, to not build; and we could
> allow to do things like: --with-ctags=ctags.emacs to explicitly create
> it as ctags.emacs... But I am sure such method will break some standard
> or development agreement.

Way too complicated IMHO.

Re: etags name collision.

[Alfred M. Szmidt]

   >Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
   >ctags is compatible with the Emacs ctags output (which is why it
   >exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
   >mg(1) on those systems, and their output is at odds with what is
   >standardized by POSIX.

   In what sense it is not standarized by POSIX?

The output from Exuberant ctags is not what POSIX asks for.  Programs
that expect the POSIX format will thus not work, or do funny stuff.

   >So really, you're suggesting to remove a standardized utility and
   >replace it with non-standard ones that produce incompatible output
   >from what is generally accepted.

   I am suggesting to avoid the forced installation of a utility that we
   are not maintaining very well for another very well maintained, with
   more languages, support and formats.

That is changing the point I was making, that Emacs ctags produces the
standard ctags output where Exuberant ctags does not.

But, I do think that the Emacs maintainers will disagree that
ctags/etags is not being maintained.  Like everything in Emacs,
someone has to do the work, and you're most welcome to send patches to
improve ctags (or etags).  We all would appreciate it.

Re: etags name collision.

[Alfred M. Szmidt]

   > I use OpenBSD in my servers daily and all my team around use vim and
   > neovim with Universal Ctags with no issue.  Universal ctags can generate
   > emacs tags, json, legacy and modern ctags, xref... if there is some
   > problem the right to do is to open an issue in their project...

   vim and neovim are not vi and mg, and are not standardized utilities
   either.

vi(1) (and ex(1) which also accepts tags files) is standardized by
POSX, FWIW

Re: etags name collision.

[Alfred M. Szmidt]

   > Are you saying vi or mg, as opposed to vim and neovim, cannot work
   > with the tags output by universal ctags?

   I'm not, but Alfred evidently is.

Can we please not assume we know what people are "evidently" saying?
Unless you are evidently me....

What I think I said was that ctags has a standardized format by POSIX,
this is what Emacs ctags (and BSD ctags) output; Exuberant ctags does
not output this format.

vi(1) (a POSIX standardized utlity, and ex(1) for that matter) and
mg(1), plus other tools expect the POSIX variant of ctags output.

Re: etags name collision.

[Dmitry Gutov]

On 12.04.2022 16:45, Alfred M. Szmidt wrote:
> What I think I said was that ctags has a standardized format by POSIX,
> this is what Emacs ctags (and BSD ctags) output; Exuberant ctags does
> not output this format.

Wikipedia says:

Extended Ctags
This is the format used by Vim's Exuberant Ctags and Universal Ctags. 
These programs can generate an original ctags file format or an extended 
format that attempts to retain backward compatibility.

Universal Ctags's manual says:

u-ctags, e-ctags u-ctags is the default output format extending the 
Exuberant Ctags output format
(e-ctags).
--format=1 and --format=2 are same as --output-format=e-ctags and
--output-format=u-ctags respectively.
See man page tags (5) for details. The difference between u-ctags and 
e-ctags are marked as “EX-
CEPTION”. Additional changes in Universal Ctags are described in Changes 
to the tags file format.

Somebody should experiment with the --format flag, but if there is 
indeed no option for the original format, 
https://en.wikipedia.org/wiki/Ctags should be updated.

Re: etags name collision.

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 12.04.2022 16:45, Alfred M. Szmidt wrote:
>> What I think I said was that ctags has a standardized format by POSIX,
>> this is what Emacs ctags (and BSD ctags) output; Exuberant ctags does
>> not output this format.
>
> Wikipedia says:
>
> Extended Ctags
> This is the format used by Vim's Exuberant Ctags and Universal Ctags.
> These programs can generate an original ctags file format or an
> extended format that attempts to retain backward compatibility.
>
> Universal Ctags's manual says:
>
> u-ctags, e-ctags u-ctags is the default output format extending the
> Exuberant Ctags output format
> (e-ctags).
> --format=1 and --format=2 are same as --output-format=e-ctags and
> --output-format=u-ctags respectively.
> See man page tags (5) for details. The difference between u-ctags and
> e-ctags are marked as “EX-
> CEPTION”. Additional changes in Universal Ctags are described in
> Changes to the tags file format.
>
> Somebody should experiment with the --format flag, but if there is
> indeed no option for the original format,
> https://en.wikipedia.org/wiki/Ctags should be updated.

FWIW: AFAIK Exhuberant Ctags is unmaintained. Now the action happens in
Universal Ctags.

Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 09:45:03AM -0400, Alfred M. Szmidt wrote:
>   >Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
>   >ctags is compatible with the Emacs ctags output (which is why it
>   >exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
>   >mg(1) on those systems, and their output is at odds with what is
>   >standardized by POSIX.
>
>   In what sense it is not standarized by POSIX?
>
>The output from Exuberant ctags is not what POSIX asks for.  Programs
>that expect the POSIX format will thus not work, or do funny stuff.
>
Again, I haven't observed that, that's exactly my point, if there is an
issue in universal ctags output format it should be reported, but I
haven't observed any mismatch.

>   >So really, you're suggesting to remove a standardized utility and
>   >replace it with non-standard ones that produce incompatible output
>   >from what is generally accepted.
>
>   I am suggesting to avoid the forced installation of a utility that we
>   are not maintaining very well for another very well maintained, with
>   more languages, support and formats.
>
>That is changing the point I was making, that Emacs ctags produces the
>standard ctags output where Exuberant ctags does not.
>
Universal ctags DOES generate standard ctags format (and some others
like etags format); they actually explicitly refer to vi in the man
page.

Exuberant is not maintained since 2011.

>But, I do think that the Emacs maintainers will disagree that
>ctags/etags is not being maintained.  Like everything in Emacs,
>someone has to do the work,

> and you're most welcome to send patches to
>improve ctags (or etags).  We all would appreciate it.
>
If there is the free and active universal ctags with a big and active
team doing that work and supporting many format and languages, then it
is pointless to duplicate effort and invest time on that.  Don't
reinvent the wheel.

Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 05:29:50PM +0300, Dmitry Gutov wrote:
>On 12.04.2022 16:45, Alfred M. Szmidt wrote:
>>What I think I said was that ctags has a standardized format by POSIX,
>>this is what Emacs ctags (and BSD ctags) output; Exuberant ctags does
>>not output this format.
>
>Wikipedia says:
>
>Extended Ctags
>This is the format used by Vim's Exuberant Ctags and Universal Ctags. 
>These programs can generate an original ctags file format or an 
>extended format that attempts to retain backward compatibility.
>
>Universal Ctags's manual says:
>
>u-ctags, e-ctags u-ctags is the default output format extending the 
>Exuberant Ctags output format
>(e-ctags).
>--format=1 and --format=2 are same as --output-format=e-ctags and
>--output-format=u-ctags respectively.
>See man page tags (5) for details. The difference between u-ctags and 
>e-ctags are marked as “EX-
>CEPTION”. Additional changes in Universal Ctags are described in 
>Changes to the tags file format.
>
>Somebody should experiment with the --format flag, but if there is 
>indeed no option for the original format, 
>https://en.wikipedia.org/wiki/Ctags should be updated.

Hi Dmitry:

The Universal ctags manual says:

--format=(1|2)
        Change the format of the output tag file. Currently the only
        valid values for level are 1 or 2. Level 1 specifies the original
        tag file format and level 2 specifies a new extended format
        containing extension fields (but in a manner which retains
        backward-compatibility with original vi(1) implementations). The
        default level is 2.  [Ignored in etags mode]

So if this 1 doesn't generate a vi compatible format it is not a
wikipedia issue, it is an issue to report.  As I said before, After a
basic test with my limited vi knowledge the format is compatible (and
BTW it worked out of the box with zero effort configuring vi.)

Re: etags name collision.

[Alfred M. Szmidt]

   >   >Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
   >   >ctags is compatible with the Emacs ctags output (which is why it
   >   >exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
   >   >mg(1) on those systems, and their output is at odds with what is
   >   >standardized by POSIX.
   >
   >   In what sense it is not standarized by POSIX?
   >
   >The output from Exuberant ctags is not what POSIX asks for.  Programs
   >that expect the POSIX format will thus not work, or do funny stuff.

   Again, I haven't observed that, that's exactly my point, if there is an
   issue in universal ctags output format it should be reported, but I
   haven't observed any mismatch.

I have observed the case, the POSIX format is stricter, the output
from ctags as specified by POSIX is (there are slight variants, but it
is always three fields):

  "%s\t%s\t/%s/\n", <identifier>, <filename>, <pattern>

for example, GNU Emacs ctags outputs:

  syms_of_cmds	cmds.c	/^syms_of_cmds (void)$/
 
where Exuberant (and Universal) ctags outputs:

  syms_of_cmds	cmds.c	/^syms_of_cmds (void)$/;"	f	typeref:typename:void

It is fair to fail on not being able to accept the above, or treating
the third field as one.  And one might also question how to treat a
pattern with a trailing double quote.

   >   >So really, you're suggesting to remove a standardized utility and
   >   >replace it with non-standard ones that produce incompatible output
   >   >from what is generally accepted.
   >
   >   I am suggesting to avoid the forced installation of a utility that we
   >   are not maintaining very well for another very well maintained, with
   >   more languages, support and formats.
   >
   >That is changing the point I was making, that Emacs ctags produces the
   >standard ctags output where Exuberant ctags does not.

   Universal ctags DOES generate standard ctags format (and some others
   like etags format); they actually explicitly refer to vi in the man
   page.

No, it does not.  See above; from Universal ctags from 2019.

   Exuberant is not maintained since 2011.

People still use software that is older, just that something is "not
maintained" (which can mean many things) doesn't mean that it stops
being useless.

   If there is the free and active universal ctags with a big and active
   team doing that work and supporting many format and languages, then it
   is pointless to duplicate effort and invest time on that.  Don't
   reinvent the wheel.

That is quite a dismissive comment, Emacs ctags existed long before
any of those existed (and at that point, we did not have any ctags on
GNU) so one might question who is reinventing the wheel here...

Re: etags name collision.

[Ergus]

On Tue, Apr 12, 2022 at 01:21:53PM -0400, Alfred M. Szmidt wrote:
>   >   >Not all systems use Exuberant Ctags or Universal Ctags.  On the BSDs,
>   >   >ctags is compatible with the Emacs ctags output (which is why it
>   >   >exists, AFAIR).  Exuberant Ctags etc do not work with either vi(1) or
>   >   >mg(1) on those systems, and their output is at odds with what is
>   >   >standardized by POSIX.
>   >
>   >   In what sense it is not standarized by POSIX?
>   >
>   >The output from Exuberant ctags is not what POSIX asks for.  Programs
>   >that expect the POSIX format will thus not work, or do funny stuff.
>
>   Again, I haven't observed that, that's exactly my point, if there is an
>   issue in universal ctags output format it should be reported, but I
>   haven't observed any mismatch.
>
>I have observed the case, the POSIX format is stricter, the output
>from ctags as specified by POSIX is (there are slight variants, but it
>is always three fields):
>
>  "%s\t%s\t/%s/\n", <identifier>, <filename>, <pattern>
>
>for example, GNU Emacs ctags outputs:
>
>  syms_of_cmds	cmds.c	/^syms_of_cmds (void)$/
>
>where Exuberant (and Universal) ctags outputs:
>
>  syms_of_cmds	cmds.c	/^syms_of_cmds (void)$/;"	f	typeref:typename:void
>
Did you tried the format=1 option?

>It is fair to fail on not being able to accept the above, or treating
>the third field as one.  And one might also question how to treat a
>pattern with a trailing double quote.
>
>   >   >So really, you're suggesting to remove a standardized utility and
>   >   >replace it with non-standard ones that produce incompatible output
>   >   >from what is generally accepted.
>   >
>   >   I am suggesting to avoid the forced installation of a utility that we
>   >   are not maintaining very well for another very well maintained, with
>   >   more languages, support and formats.
>   >
>   >That is changing the point I was making, that Emacs ctags produces the
>   >standard ctags output where Exuberant ctags does not.
>
>   Universal ctags DOES generate standard ctags format (and some others
>   like etags format); they actually explicitly refer to vi in the man
>   page.
>
>No, it does not.  See above; from Universal ctags from 2019.
>
See above

>   Exuberant is not maintained since 2011.
>
>People still use software that is older, just that something is "not
>maintained" (which can mean many things) doesn't mean that it stops
>being useless.
>
Agree, but any error in Exuberant Ctags has been fixed in Universal
version and can be reported there to be fixed immediately.

>   If there is the free and active universal ctags with a big and active
>   team doing that work and supporting many format and languages, then it
>   is pointless to duplicate effort and invest time on that.  Don't
>   reinvent the wheel.
>
>That is quite a dismissive comment, Emacs ctags existed long before
>any of those existed (and at that point, we did not have any ctags on
>GNU) so one might question who is reinventing the wheel here...
>
Agree, but now their wheel is better. If they had the motivation, man
power and patience to implement something more complete and compatible,
in a shorter time and share it with the community for 10 years, the fact
that many users prefer it and the backward compatibility they have with
the original version... plus a lack of man power we have...  it is
pointless to duplicate the effort. Emacs developers should join effort
in other direction (treesitter, lsp, project features and so on; that's
my personal opinion of course.)

Re: etags name collision.

[Stefan Monnier]

>    If there is the free and active universal ctags with a big and active
>    team doing that work and supporting many format and languages, then it
>    is pointless to duplicate effort and invest time on that.  Don't
>    reinvent the wheel.
>
> That is quite a dismissive comment, Emacs ctags existed long before
> any of those existed (and at that point, we did not have any ctags on
> GNU) so one might question who is reinventing the wheel here...

Hmm... if you're talking about the past then Emacs, Exhuberant Ctags,
and Universal Ctags have all reinventing the wheel here.  But that's the
past, what's done is done.

Ergus was talking about current efforts, where indeed it seems that
trying to invest effort into making our etags/ctags code work as well as
Universal Ctags is a kind of reinventing the wheel and the benefit
doesn't seem very clear.


        Stefan

Re: etags name collision.

[Alfred M. Szmidt]

   >where Exuberant (and Universal) ctags outputs:
   >
   >  syms_of_cmds	cmds.c	/^syms_of_cmds (void)$/;"	f	typeref:typename:void

   Did you tried the format=1 option?

No, it is not supported by POSIX -- that is sorta the point of using
standard-ish compliant utilities.

Re: etags name collision.

[Dmitry Gutov]

On 12.04.2022 22:50, Alfred M. Szmidt wrote:
>     >where Exuberant (and Universal) ctags outputs:
>     >
>     >  syms_of_cmds	cmds.c	/^syms_of_cmds (void)$/;"	f	typeref:typename:void
> 
>     Did you tried the format=1 option?
> 
> No, it is not supported by POSIX -- that is sorta the point of using
> standard-ish compliant utilities.

What's the point, though?

I can understand trying to support a standard format, but for programs...?

Would you also standardize Emacs and avoid adding new features because 
of that?

Re: etags name collision.

[Alfred M. Szmidt]

   I can understand trying to support a standard format, but for programs...?

Programs relay on the standard format to be what it is.

   Would you also standardize Emacs and avoid adding new features because 
   of that?

Nobody said anything about avoiding new features.