From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: Generation of tags for the current project on the fly Date: Mon, 15 Jan 2018 21:50:33 +0300 Message-ID: <98f4f0c3-6815-bf86-fa23-1a330c60b9f3@yandex.ru> References: <4559858d-eb32-d071-fdad-e51430700260@yandex.ru> <83shbb30z1.fsf@gnu.org> <8360863o6a.fsf@gnu.org> <27a58fb2-d2ee-e5fc-158d-ec41be401987@yandex.ru> <83y3l0za1f.fsf@gnu.org> <259c557d-e3a3-c01b-9ba3-30df09d247ea@yandex.ru> <83inc3znpu.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1516042149 27491 195.159.176.226 (15 Jan 2018 18:49:09 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 15 Jan 2018 18:49:09 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Thunderbird/58.0 Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jan 15 19:49:04 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eb9oQ-0005vT-Dq for ged-emacs-devel@m.gmane.org; Mon, 15 Jan 2018 19:48:50 +0100 Original-Received: from localhost ([::1]:54787 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eb9qQ-0003qx-87 for ged-emacs-devel@m.gmane.org; Mon, 15 Jan 2018 13:50:54 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:38918) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eb9qD-0003pF-IZ for emacs-devel@gnu.org; Mon, 15 Jan 2018 13:50:42 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eb9q9-0008B5-KH for emacs-devel@gnu.org; Mon, 15 Jan 2018 13:50:41 -0500 Original-Received: from mail-lf0-x231.google.com ([2a00:1450:4010:c07::231]:41702) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eb9q9-0008Ah-AH; Mon, 15 Jan 2018 13:50:37 -0500 Original-Received: by mail-lf0-x231.google.com with SMTP id h137so14491891lfe.8; Mon, 15 Jan 2018 10:50:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=JRxD8as8/MnwUyCU7GryOYPymf1F3ORshSGzmqAr6n4=; b=NC16MHNRco1AqlxM4z+uzhMsSQPNarUFlHtIZ5tPqAkD1PZxAqapS1Q97dTTy2EJk/ 7Uid6+uXFvkIEgTi+32+Yi1AyWLdElpqIo7hvdH8bEGzu+66F3sJCsJmhCV0M90kdepl UAA1qn9c5r/KuA92ZdqU9nBUKtFVs/yWEjuFFoJ/Gz3xiW30+d0C6Is+Bz6OHOrYODof /gNjTBpTEATmMZjd8g77LkxC2lXVbRcc0zLB+o31zBI2adPHszX2rEHbm/TN/HnCy0L/ 76jMa2HzFnBdoPYaUL7m3cwlYphJ+BcZftzwP1+Mra/g9EmzQ2kYrGoLmZ8DTJKws7ix wnTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=JRxD8as8/MnwUyCU7GryOYPymf1F3ORshSGzmqAr6n4=; b=HGocfZ7kz6m4LhptxrLxAeJt923hBR8RxqGA4x9rz3h/jTRixxqWpwApVvUg3J/Zyy eHQMeyewkP7FbdDka8JhPDSIh5TJXM8UDebpgSPac5KmAxiHdPdl375141CPWmbpB5+X zGOH0KNMBS8xgNQwBeDlA/rzeBKIBo6O+0skeNn5c7QobFNfnvIsWeTJuuKjgYEX4mKR 0nmUkPBqz0LNfH3Dv2ZqOFlrifYb50DBTzsPgxNdAKm2tsbOcqA6Ydq8q90gjvTQOt8Q 7Ql1P5uVrIq4hu2wH2gXhkeuSTvPsLBH7CgKHdC3/KnCP2e3/UebIoQJqLhiu6B0K4lA lPTA== X-Gm-Message-State: AKwxyteUxGu9/elDkVixrfzCWjYDSSYjET7gUDDLQ+ATMwUnWUGs6B3u 2WQ03MG0lGXUbDdjkH0A0unyfa4r X-Google-Smtp-Source: ACJfBov0vwWUOjrBJX1VhdUoVtFoZzh4e1pYpT2LJmcgPtbXwoG2s/FtY6JvcDOCTUzXfrDoLiObjQ== X-Received: by 10.25.42.196 with SMTP id q65mr11638372lfq.133.1516042235374; Mon, 15 Jan 2018 10:50:35 -0800 (PST) Original-Received: from [192.168.1.174] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id k77sm47313lfe.48.2018.01.15.10.50.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 15 Jan 2018 10:50:34 -0800 (PST) In-Reply-To: <83inc3znpu.fsf@gnu.org> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4010:c07::231 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:221980 Archived-At: On 1/15/18 8:37 AM, Eli Zaretskii wrote: >>> No, I think asking once per project should be enough. >> >> Until the end of the current Emacs session? And ask again after restart? > > Yes. > >> What about if the user switches to a different project and then back? > > Ideally, don't ask anymore about that project. This is doable, ok. >> There's also another optimization opportunity: performing reindexing in >> an asynchronous fashion, in the background (maybe after a timeout, too), >> after any file is changed and saved. This one comes with its own >> tradeoffs, though. > > Doing that asynchronously could be an automatic action , not in need > of any user confirmation. It complicates the implementation a bit, > but perhaps not too much, so this could be a good design choice. It would shorten the waits, but do nothing for the CPU and disk usage. Those I'm more worried about, actually, as a laptop user with a not-so-great battery life on GNU/Linux. If we just invalidate on save, the rescan doesn't happen until you intend to use it again. And with asynchronous approach, they will occur again and again, just as you edit and save files. With large projects, one CPU core might always be busy this way (or does etags parallelize? more cores then). So maybe someone would prefer this approach, but I'd only go for it only as a qualify-of-life improvements when scans are already pretty short. >> To measure the full time: >> >> (benchmark 1 '(progn (etags--project-tags-cleanup) >> (etags--maybe-use-project-tags))) > > 5.5 sec with warm cache. This is with an unoptimized Emacs, btw, but > most of the time is spent by external programs, so perhaps this > doesn't matter. Probably doesn't, indeed. >> To measure the time to generate the list of files only: >> >> (benchmark 1 '(all-completions "" (project-file-completion-table >> (project-current) (list default-directory)))) > > 0.95 sec with cold cache, 0.23 with warm cache. Thanks, so 1 second for file listing for 4.5 seconds for etags. Even if we allowed etags to execute in parallel with find, it could only shave it down to 4.5 seconds (and probably not even that). >> How do we figure which files to visit? Do we just visit src/TAGS and >> expect the rest to be 'include'-d. > > I think just visit TAGS in the directory of the source whose symbol is > requested, or maybe use locate-dominating-file to look higher in the > tree if not found in the current directory. That option is not as easy to code as what I suggested. Further, if we just visit lisp/TAGS when in lisp/, and xref-etags-mode is enabled, we won't be able to find the definition of 'car'. >> Here's another one: considering the reindexing costs are not always >> negligible and depend on the size of a project, will there be actual >> benefit to using the proposed scheme in GNU projects like Emacs, GCC and >> others (those are the ones that use 'make TAGS')? Or is there a subset >> of them, at least, which we expect to benefit? > > That's a good question. But if the tags table is automatically > produced in the background, the time this takes is much less > important, and having TAGS always up to date would be a valuable > feature. FWIW, I do "make TAGS" in every large project I start > working on seriously, so at least for me this is important. You probably do it just once, though, and update very rarely. The old way of operation is still going to work. Can we improve the "warm" reindex times? In the first message of this thread I mentioned GNU Global because it reportedly supports incremental updates. Can we get such feature in etags, too? I more or less imagine how I'd implement such a feature using Lisp and 'etags --append', but that would do nothing to help when the tags are generated by make.