Re: Generation of tags for the current project on the fly

[Eli Zaretskii <eliz@gnu.org>]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 15 Jan 2018 04:44:58 +0300
> 
> On 1/14/18 7:21 PM, Eli Zaretskii wrote:
> 
> >> OK, so if the user says yes, we "temporarily visit" to auto-generated
> >> tags table. Then the user saves a file and that table get invalidated
> >> (or via some other mechanism), and we want to index it again. Ask again?
> > 
> > No, I think asking once per project should be enough.
> 
> Until the end of the current Emacs session? And ask again after restart?

Yes.

> What about if the user switches to a different project and then back?

Ideally, don't ask anymore about that project.

> > I mean the first time the tags table is required might very well be at
> > the beginning of working on a project, at which time the project
> > source tree is not yet in the cache.
> 
> Yes, and? The user will need it to be indexed either way, right?

Yes, but my point was that col-cache times cannot be ignored.

> There's also another optimization opportunity: performing reindexing in 
> an asynchronous fashion, in the background (maybe after a timeout, too), 
> after any file is changed and saved. This one comes with its own 
> tradeoffs, though.

Doing that asynchronously could be an automatic action , not in need
of any user confirmation.  It complicates the implementation a bit,
but perhaps not too much, so this could be a good design choice.

> To measure the full time:
> 
> (benchmark 1 '(progn (etags--project-tags-cleanup) 
> (etags--maybe-use-project-tags)))

5.5 sec with warm cache.  This is with an unoptimized Emacs, btw, but
most of the time is spent by external programs, so perhaps this
doesn't matter.

> To measure the time to generate the list of files only:
> 
> (benchmark 1 '(all-completions "" (project-file-completion-table 
> (project-current) (list default-directory))))

0.95 sec with cold cache, 0.23 with warm cache.

> >> 'git ls-files' will probably be faster still.
> > 
> > Yes, but that only works in Git repositories.
> 
> We can probably optimize for that use case these days. Git or some other 
> VCS is usually in place, especially in non-toy projects.

For the projects using Git, yes.

> How do we figure which files to visit? Do we just visit src/TAGS and 
> expect the rest to be 'include'-d.

I think just visit TAGS in the directory of the source whose symbol is
requested, or maybe use locate-dominating-file to look higher in the
tree if not found in the current directory.

> Here's another one: considering the reindexing costs are not always 
> negligible and depend on the size of a project, will there be actual 
> benefit to using the proposed scheme in GNU projects like Emacs, GCC and 
> others (those are the ones that use 'make TAGS')? Or is there a subset 
> of them, at least, which we expect to benefit?

That's a good question.  But if the tags table is automatically
produced in the background, the time this takes is much less
important, and having TAGS always up to date would be a valuable
feature.  FWIW, I do "make TAGS" in every large project I start
working on seriously, so at least for me this is important.