From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: Automatic (e)tags generation and incremental updates Date: Sun, 10 Jan 2021 15:53:10 +0200 Message-ID: <1e9c9572-52ee-339b-78a2-731b9eb5f3de@yandex.ru> References: <779a6328-9ca5-202a-25a2-b270c66fe6dd@yandex.ru> <8fc5e96c-ebb8-c668-9b2a-c7c4ee54c0b9@yandex.ru> <83r1mwltob.fsf@gnu.org> <0bee9ab4-46bc-b6fd-97b6-e26cc80f1610@yandex.ru> <875z45dbm7.fsf@tromey.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8960"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 Cc: Eli Zaretskii , john@yates-sheets.org, philipk@posteo.net, emacs-devel@gnu.org To: Tom Tromey Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jan 10 14:54:14 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kybAc-0002EB-Q8 for ged-emacs-devel@m.gmane-mx.org; Sun, 10 Jan 2021 14:54:14 +0100 Original-Received: from localhost ([::1]:57820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kybAb-0003a6-Uu for ged-emacs-devel@m.gmane-mx.org; Sun, 10 Jan 2021 08:54:13 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kyb9m-00037v-P2 for emacs-devel@gnu.org; Sun, 10 Jan 2021 08:53:22 -0500 Original-Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]:53253) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kyb9h-00062C-M0; Sun, 10 Jan 2021 08:53:22 -0500 Original-Received: by mail-wm1-x329.google.com with SMTP id k10so11625071wmi.3; Sun, 10 Jan 2021 05:53:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=EkgswiNZlweYc1L0GMHRvUulmF6zXy4GNv7fAr5OwHY=; b=aLWfBXXfwIl/KIOwWtNIlQ6y1f2KEKco+nnJeNNyYwJK12BhPOw7Dmio0M9ZL6FNuB qPrYy3ZLXh9Qk/u5QVxPPNyBU6WhcsOOWYm1Phe3L76nY9x5pJdnnPV0ooY1jr/kFCmA YYtWqim3sifAhYEk/uxrHHxV0PZuPIKKFsi4CTOxnXNQUz5N9CthEXRjBI756dVQ5/8w x3B93CemkBU0WaCZwj3u2TRlbp5C3KZu4ZP0heqKU86vOunyZ30eCYBh6MfP3xIITa+W k+NS01TUr91cIfLw/feZK1ZvBaGECYysqJVNA9wJAEeqx/2o/7f+/JASpyCftCwXaW7N c+SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=EkgswiNZlweYc1L0GMHRvUulmF6zXy4GNv7fAr5OwHY=; b=uVJ5GUfN0gxcplR1bZZsEBwkVuS/+wDGow5gF9y89TYGxCDU4mafdmNJO0sIKv3ZFa e6c4MvK0TMF+7Qrmmbis/nEtXwfYYz4vne+54dtLTyVU9Pc9q3fFqoo9NxBXjVLFh4b5 gwkt96M5xIyJbCDejVSa/VgV0vc5v/3DZcj58MZdJRgqdk+qMMPuBclNP7BAetTdQYwx ZC2HeBHukry42pnaaPIS6mXURdo4CRO6AbPeM1ntCLDbL9/wtYA+F2dgu/KO6OUQngxz XVbarGhX6pyTMjnoK6E17CZH5TOyZMkaXuHxGxxTIH9Zyys5nIoOnAoVNRTZRATj5RDY NAcQ== X-Gm-Message-State: AOAM531BpjM1aYqmeLtGw0Jczochxk29hDYSrzWtivJmDLOaS4PwTN9/ KpikkDk5PK0wm3dwu3U9OWg= X-Google-Smtp-Source: ABdhPJwF0sOJzGQ0lHH0sGILsiGT+nSwdy81KHnSCU98noMPiUbjHIG8OrgKWuiUmQQbeWBoQ6vmOg== X-Received: by 2002:a1c:9d16:: with SMTP id g22mr10732371wme.140.1610286794848; Sun, 10 Jan 2021 05:53:14 -0800 (PST) Original-Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id b7sm20026721wru.33.2021.01.10.05.53.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 10 Jan 2021 05:53:13 -0800 (PST) In-Reply-To: <875z45dbm7.fsf@tromey.com> Content-Language: en-US Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=raaahh@gmail.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, NICE_REPLY_A=-0.012, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:262841 Archived-At: Hi Tom, On 09.01.2021 23:49, Tom Tromey wrote: > Dmitry> - Call some xref command like 'M-.'. See message "Generating new tags > Dmitry> table..." (it's saved in /tmp/...), then see navigation happen. > > On "xdg"-ish systems, it would make sense to use the XDG cache directory > here. I should look into this later. If we end up using putting TAGS files into prject directories (see below), that will be moot. > Dmitry> - Pressing C-M-i instead should also trigger tags table generation. > > Dmitry> - When you switch between projects, the previously generated tags > Dmitry> tables are discarded. It's not too hard to improve, but that would > Dmitry> involve some choices/tradeoffs. > > What are the tradeoffs? There are several questions. - Do we want to store the generated files openly in the root directories of each project? I.e. save them as TAGS. That might look more familiar/comfortable by the old-timers, and some users might even pre-generate such files, if the process takes a long time. And the contents will be able to reliably survive for a long time. Storing them with "garbled" names somewhere in /tmp of XDG cache risks having to fully renenerate the indexes at least every time the machine reboots. The downside to creating TAGS files is it's unfamiliar to newcomers who usually expect index cache to be hidden, and they'll have to either update .gitignore or risk checking them in. I don't see a lot of the old timers in this discussion now, and most existing users of etags are likely satisfied with the current workflow, so perhaps that kind of familiarity is not important. So it probably comes down to being able to generate such files only once per project, and only update them later. I'm not such what size of project that will become a significant advantage at, but it's likely that at that point etags.el's other performance limitation will come into play. We'll need some real feedback on that. Until then, the generated files will stay in /tmp. Might even keep them off disk entirely, actually, though that would require some changes to etags.el (help welcome). - Do we keep such file in memory every time after the user has switched to a different project (and, say, maintain a {project -> file} alist in memory), or close and reopen upon switching. If the files are stored on disk, implementing the latter is plainly easier with the current etags.el code. Is eliminating the delay worth the code complexity and increased memory usage? - Being able to pick up an old TAGS depends on our ability to compare the current project contents against a list of files and one timestamp (TAGS modification datetime) quickly, much quicker than simply regenerating such file would take. And if we can't, there's no point in keeping them around. > I tend to think that conceptually each buffer should point to its > corresponding tags table. Then some separate logic could be used to > decide when to kill some tag file buffer. Ideally, perhaps, etags.el would provide an interface for polling a specific tags table (for completions or locations) by simply binding one or two local variables. At the moment, though, the route there seems to be through file-local variables and through calling visit-tags-table with non-nil second argument. > Dmitry> - When files are deleted, or otherwise changed outside of Emacs > Dmitry> (perhaps with 'git checkout'), nothing is updated. I have a few new > Dmitry> ideas, haven't started on them yet. Workaround: toggle > Dmitry> etags-regen-mode off and on, which will result in full rescan when you > Dmitry> use 'M-.'. > > It seems to me that the default ought to be to update the tags table on M-. > One nice way to do this would be to run etags in the background, so that > the work of updating would be done in parallel with the user typing, > since presumably you'd want to ensure that etags has finished before > jumping to the result (or fulfilling a completion request). It sounds clever, but UUIC that would only benefit users who call M-. with C-u or who have customized xref-prompt-for-identifier to t. Even among those, it would only be able to help (without sacrificing correctness) only those who don't use something like icomplete-mode for tag input. Because the completion table already depends on the tags index (which is out of date). On the flip side, even when the tags generation is synchronous, you can start typing right away. If general, doing updates when Emacs is idle and/or asynchronously are quality-of-life changes that can come later after we improve correctness (i.e. make sure the index is up to date even after external changes). Doing that will require some more processing implemented in Elisp which can still create annoying stutters in Emacs, whether the process calls are asynchronous or not. Debugging those and working on algorithmic complexity there is easier when work happens at predictable points in time. Asynchronous calls also make error handling more difficult (and the current Emacs threads -- even more so).