[PATCH] build: generate cscope and etags source indexes

[PATCH] build: generate cscope and etags source indexes

[Yuri Volchkov]

Just a handy tool to generate source code indexes for your favorite
editor.

To make tags for vim run:
  $ make cscope
To make tags for emacs:
  $ make TAGS
To make both:
  $ make alltags

Signed-off-by: Yuri Volchkov <yuri.volchkov@gmail.com>
---
 Makefile.local | 24 ++++++++++++++++++++++++
 configure      | 30 ++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/Makefile.local b/Makefile.local
index af12ca7..15075bb 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -280,6 +280,30 @@ endif
 SRCS  := $(SRCS) $(notmuch_client_srcs)
 CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules)
 CLEAN := $(CLEAN) version.stamp notmuch-*.tar.gz.tmp
+CLEAN := $(CLEAN) cscope.out TAGS
+
+.PHONY: alltags cscope
+alltags: cscope.out TAGS
+cscope: cscope.out
+
+ifeq ($(HAVE_CSCOPE),1)
+cscope.out: $(SRCS)
+	$(call quiet, CSCOPE) -b $(SRCS)
+else
+.PHONY: cscope.out
+cscope.out:
+	$(warning "$@ is not generated. Please make sure you have cscope installed")
+endif
+
+ifeq ($(HAVE_ETAGS),1)
+TAGS: $(SRCS)
+	@rm -f $@
+	$(call quiet, ETAGS) -a $(SRCS)
+else
+.PHONY: TAGS
+TAGS:
+	$(warning "$@ is not generated. Please make sure you have etags installed")
+endif
 
 DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config
 
diff --git a/configure b/configure
index c5e2ffe..4467016 100755
--- a/configure
+++ b/configure
@@ -603,6 +603,24 @@ else
     have_emacs=0
 fi
 
+printf "Checking if cscope is available... "
+if cscope -V > /dev/null 2>&1; then
+    printf "Yes.\n"
+    have_cscope=1
+else
+    printf "No (so will not generate cscope code index - file cscope.out)\n"
+    have_cscope=0
+fi
+
+printf "Checking if etags is available... "
+if etags -V > /dev/null 2>&1; then
+    printf "Yes.\n"
+    have_etags=1
+else
+    printf "No (so will not generate etags code index - file TAGS)\n"
+    have_etags=0
+fi
+
 have_doxygen=0
 if [ $WITH_API_DOCS = "1" ] ; then
     printf "Checking if doxygen is available... "
@@ -1011,6 +1029,12 @@ WARN_CXXFLAGS=${WARN_CXXFLAGS}
 # Flags to enable warnings when using the C compiler
 WARN_CFLAGS=${WARN_CFLAGS}
 
+# Command to execute cscope from Makefiles
+CSCOPE = cscope
+
+# Command  to execute etags from Makefiles
+ETAGS = etags
+
 # Name of python interpreter
 PYTHON = ${python}
 
@@ -1046,6 +1070,12 @@ emacsetcdir=${EMACSETCDIR}
 # Whether there's an emacs binary available for byte-compiling
 HAVE_EMACS = ${have_emacs}
 
+# Whether there's a cscope available for source code indexing (development only)
+HAVE_CSCOPE = ${have_cscope}
+
+# Whether there's a etags available for source code indexing (development only)
+HAVE_ETAGS = ${have_etags}
+
 # Whether there's a sphinx-build binary available for building documentation
 HAVE_SPHINX=${have_sphinx}
 
-- 
2.7.4

Re: [PATCH] build: generate cscope and etags source indexes

[Jani Nikula]

On Wed, 23 Aug 2017, Yuri Volchkov <yuri.volchkov@gmail.com> wrote:
> Just a handy tool to generate source code indexes for your favorite
> editor.
>
> To make tags for vim run:
>   $ make cscope
> To make tags for emacs:
>   $ make TAGS
> To make both:
>   $ make alltags

What's the point in adding these to configure? Or, to be honest, to the
build at all?

In theory you'll be able to look at $(SRCS) for indexing... but those
are only the .c/.cc files. Are your tools clever enough to follow
#include directives to index the headers as well?

I guess I'm also biased because I use gnu global [1] instead. And for
that I have a script of my own that basically boils down to:

$ git ls-files | gtags -f -

which means all files tracked by git get indexed, and global will decide
whether it can index the file or not.

BR,
Jani.

[1] https://www.gnu.org/software/global/

>
> Signed-off-by: Yuri Volchkov <yuri.volchkov@gmail.com>
> ---
>  Makefile.local | 24 ++++++++++++++++++++++++
>  configure      | 30 ++++++++++++++++++++++++++++++
>  2 files changed, 54 insertions(+)
>
> diff --git a/Makefile.local b/Makefile.local
> index af12ca7..15075bb 100644
> --- a/Makefile.local
> +++ b/Makefile.local
> @@ -280,6 +280,30 @@ endif
>  SRCS  := $(SRCS) $(notmuch_client_srcs)
>  CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules)
>  CLEAN := $(CLEAN) version.stamp notmuch-*.tar.gz.tmp
> +CLEAN := $(CLEAN) cscope.out TAGS
> +
> +.PHONY: alltags cscope
> +alltags: cscope.out TAGS
> +cscope: cscope.out
> +
> +ifeq ($(HAVE_CSCOPE),1)
> +cscope.out: $(SRCS)
> +	$(call quiet, CSCOPE) -b $(SRCS)
> +else
> +.PHONY: cscope.out
> +cscope.out:
> +	$(warning "$@ is not generated. Please make sure you have cscope installed")
> +endif
> +
> +ifeq ($(HAVE_ETAGS),1)
> +TAGS: $(SRCS)
> +	@rm -f $@
> +	$(call quiet, ETAGS) -a $(SRCS)
> +else
> +.PHONY: TAGS
> +TAGS:
> +	$(warning "$@ is not generated. Please make sure you have etags installed")
> +endif
>  
>  DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config
>  
> diff --git a/configure b/configure
> index c5e2ffe..4467016 100755
> --- a/configure
> +++ b/configure
> @@ -603,6 +603,24 @@ else
>      have_emacs=0
>  fi
>  
> +printf "Checking if cscope is available... "
> +if cscope -V > /dev/null 2>&1; then
> +    printf "Yes.\n"
> +    have_cscope=1
> +else
> +    printf "No (so will not generate cscope code index - file cscope.out)\n"
> +    have_cscope=0
> +fi
> +
> +printf "Checking if etags is available... "
> +if etags -V > /dev/null 2>&1; then
> +    printf "Yes.\n"
> +    have_etags=1
> +else
> +    printf "No (so will not generate etags code index - file TAGS)\n"
> +    have_etags=0
> +fi
> +
>  have_doxygen=0
>  if [ $WITH_API_DOCS = "1" ] ; then
>      printf "Checking if doxygen is available... "
> @@ -1011,6 +1029,12 @@ WARN_CXXFLAGS=${WARN_CXXFLAGS}
>  # Flags to enable warnings when using the C compiler
>  WARN_CFLAGS=${WARN_CFLAGS}
>  
> +# Command to execute cscope from Makefiles
> +CSCOPE = cscope
> +
> +# Command  to execute etags from Makefiles
> +ETAGS = etags
> +
>  # Name of python interpreter
>  PYTHON = ${python}
>  
> @@ -1046,6 +1070,12 @@ emacsetcdir=${EMACSETCDIR}
>  # Whether there's an emacs binary available for byte-compiling
>  HAVE_EMACS = ${have_emacs}
>  
> +# Whether there's a cscope available for source code indexing (development only)
> +HAVE_CSCOPE = ${have_cscope}
> +
> +# Whether there's a etags available for source code indexing (development only)
> +HAVE_ETAGS = ${have_etags}
> +
>  # Whether there's a sphinx-build binary available for building documentation
>  HAVE_SPHINX=${have_sphinx}
>  
> -- 
> 2.7.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] build: generate cscope and etags source indexes

[Tomi Ollila]

On Thu, Aug 24 2017, Jani Nikula wrote:

> On Wed, 23 Aug 2017, Yuri Volchkov <yuri.volchkov@gmail.com> wrote:
>> Just a handy tool to generate source code indexes for your favorite
>> editor.
>>
>> To make tags for vim run:
>>   $ make cscope
>> To make tags for emacs:
>>   $ make TAGS
>> To make both:
>>   $ make alltags
>
> What's the point in adding these to configure? Or, to be honest, to the
> build at all?

I agree with Jani about configure and build steps, but could tolerate
convenience goals like `cscope` and `tags` (and something `global` !)
which would build corresponding files for developers to utilize (provided
those are accurate enough, we don't want to lessen general quality (from
what it is now) of the system due to too bad quality tags files
(content-wise)...

Tomi (who should get to work with gnu global again...)

>
> In theory you'll be able to look at $(SRCS) for indexing... but those
> are only the .c/.cc files. Are your tools clever enough to follow
> #include directives to index the headers as well?
>
> I guess I'm also biased because I use gnu global [1] instead. And for
> that I have a script of my own that basically boils down to:
>
> $ git ls-files | gtags -f -

(... which means I have to try git ls-files --recurse-submodules | gtags -f -)

>
> which means all files tracked by git get indexed, and global will decide
> whether it can index the file or not.
>
> BR,
> Jani.
>
> [1] https://www.gnu.org/software/global/
>
>>
>> Signed-off-by: Yuri Volchkov <yuri.volchkov@gmail.com>

Re: [PATCH] build: generate cscope and etags source indexes

[Yuri Volchkov]

to Jani:

>> What's the point in adding these to configure? Or, to be honest, to the
>> build at all?
Ok, modifying configure was probably unnecessary.

>> I guess I'm also biased because I use gnu global [1] instead. And for
>> that I have a script of my own that basically boils down to:
>>
>> $ git ls-files | gtags -f -
I was trying to adapt developing patterns from the linux kernel, to
which I used to. This was my bias :)

The good thing about this approach is that only those files will be
indexed, which are actually build in the current configuration. For
linux it is absolutely must, because there is a huge number of
functions, implemented differently for different architecture or
configuration option. So only relevant files are getting into the
index.

However, I agree, a relatively small project as notmuch, almost does not
suffer from this problem.

>> In theory you'll be able to look at $(SRCS) for indexing... but those
>> are only the .c/.cc files. Are your tools clever enough to follow
>> #include directives to index the headers as well?
Oops. Honestly this thing have slipped from my mind. I'll fix this if we
decided this feature is needed, and if the result will not look too ugly.

to Tomi:

> I agree with Jani about configure and build steps, but could tolerate
> convenience goals like `cscope` and `tags` (and something `global` !)
> which would build corresponding files for developers to utilize (provided
> those are accurate enough, we don't want to lessen general quality (from
> what it is now) of the system due to too bad quality tags files
> (content-wise)...

Sorry, I did not get what is accurate enough, what is lessen quality and
what is too bad? Could you please explain a little more detailed?

Also I have never tried gnu global. I need to check it out too. And
again, if decided this helper is needed, I'll add gnu global too.

Re: [PATCH] build: generate cscope and etags source indexes

[Jani Nikula]

On Thu, Aug 24, 2017 at 12:13 PM, Yuri Volchkov <yuri.volchkov@gmail.com> wrote:
>>> $ git ls-files | gtags -f -
> I was trying to adapt developing patterns from the linux kernel, to
> which I used to. This was my bias :)
>
> The good thing about this approach is that only those files will be
> indexed, which are actually build in the current configuration. For
> linux it is absolutely must, because there is a huge number of
> functions, implemented differently for different architecture or
> configuration option. So only relevant files are getting into the
> index.
>
> However, I agree, a relatively small project as notmuch, almost does not
> suffer from this problem.

FWIW I use the above snippet also for kernel work. More often than not
I want to find all the references. I think this is even more so for
notmuch, where you're more likely to do project wide refactoring.

>>> In theory you'll be able to look at $(SRCS) for indexing... but those
>>> are only the .c/.cc files. Are your tools clever enough to follow
>>> #include directives to index the headers as well?
> Oops. Honestly this thing have slipped from my mind. I'll fix this if we
> decided this feature is needed, and if the result will not look too ugly.

Not that the kernel tags targets are a good example for anything, but
they do use a bunch of complicated shell scripting to find the sources
in the file system. They don't look at the sources defined by
Makefiles for the configuration options. Figuring out the header files
in Makefiles is more trouble than it's worth.

> Also I have never tried gnu global. I need to check it out too. And
> again, if decided this helper is needed, I'll add gnu global too.

Really the simplest thing for gnu global is to just run 'gtags' in the
top level directory, and let it recursively handle all files it
understands. Having to run 'make gtags' is not much of a convenience!
;)

BR,
Jani.

Re: [PATCH] build: generate cscope and etags source indexes

[Yuri Volchkov]

Jani Nikula <jani@nikula.org> writes:

> On Thu, Aug 24, 2017 at 12:13 PM, Yuri Volchkov <yuri.volchkov@gmail.com> wrote:
>>>> $ git ls-files | gtags -f -
>> I was trying to adapt developing patterns from the linux kernel, to
>> which I used to. This was my bias :)
>>
>> The good thing about this approach is that only those files will be
>> indexed, which are actually build in the current configuration. For
>> linux it is absolutely must, because there is a huge number of
>> functions, implemented differently for different architecture or
>> configuration option. So only relevant files are getting into the
>> index.
>>
>> However, I agree, a relatively small project as notmuch, almost does not
>> suffer from this problem.
>
> FWIW I use the above snippet also for kernel work. More often than not
> I want to find all the references. I think this is even more so for
> notmuch, where you're more likely to do project wide refactoring.
>
>>>> In theory you'll be able to look at $(SRCS) for indexing... but those
>>>> are only the .c/.cc files. Are your tools clever enough to follow
>>>> #include directives to index the headers as well?
>> Oops. Honestly this thing have slipped from my mind. I'll fix this if we
>> decided this feature is needed, and if the result will not look too ugly.
>
> Not that the kernel tags targets are a good example for anything, but
> they do use a bunch of complicated shell scripting to find the sources
> in the file system. They don't look at the sources defined by
> Makefiles for the configuration options. Figuring out the header files
> in Makefiles is more trouble than it's worth.
Well, for my needs kernel tags doing pretty good job. Partially because
I use helm-git-grep whenever I need project-wide refactoring.

But I agree, that machinery for building index is way too complicated in
the kernel. That's why I said "not too ugly".
>
>> Also I have never tried gnu global. I need to check it out too. And
>> again, if decided this helper is needed, I'll add gnu global too.
>
> Really the simplest thing for gnu global is to just run 'gtags' in the
> top level directory, and let it recursively handle all files it
> understands. Having to run 'make gtags' is not much of a convenience!
> ;)

I feel convinced. Just forget about this patch.
>
> BR,
> Jani.