From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Spencer Baugh Newsgroups: gmane.emacs.bugs Subject: bug#62837: [PATCH] Add a semantic-symref backend which uses xref-matches-in-files Date: Tue, 18 Apr 2023 21:26:13 -0400 Message-ID: References: <5e6eddd5-4b38-5765-05f3-dd6c1927edd3@yandex.ru> <871qkkn720.fsf@catern.com> <37ee089e-5c42-15c6-c8c1-48642bf4e180@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="23857"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: sbaugh@catern.com, 62837@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Apr 19 03:27:11 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1powbH-000628-JK for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 19 Apr 2023 03:27:11 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1powb9-0007zf-Fe; Tue, 18 Apr 2023 21:27:03 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1powb8-0007zS-CY for bug-gnu-emacs@gnu.org; Tue, 18 Apr 2023 21:27:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1powb8-0000SQ-4X for bug-gnu-emacs@gnu.org; Tue, 18 Apr 2023 21:27:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1powb8-0006z1-0n for bug-gnu-emacs@gnu.org; Tue, 18 Apr 2023 21:27:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Spencer Baugh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 19 Apr 2023 01:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62837 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 62837-submit@debbugs.gnu.org id=B62837.168186758826793 (code B ref 62837); Wed, 19 Apr 2023 01:27:01 +0000 Original-Received: (at 62837) by debbugs.gnu.org; 19 Apr 2023 01:26:28 +0000 Original-Received: from localhost ([127.0.0.1]:60748 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1powaY-0006y4-BG for submit@debbugs.gnu.org; Tue, 18 Apr 2023 21:26:28 -0400 Original-Received: from mxout5.mail.janestreet.com ([64.215.233.18]:35583) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1powaT-0006xl-8N for 62837@debbugs.gnu.org; Tue, 18 Apr 2023 21:26:25 -0400 In-Reply-To: <37ee089e-5c42-15c6-c8c1-48642bf4e180@yandex.ru> (Dmitry Gutov's message of "Wed, 19 Apr 2023 04:10:24 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:260254 Archived-At: Dmitry Gutov writes: > On 16/04/2023 00:56, sbaugh@catern.com wrote: > >>> Perhaps you could describe your case where you *did* see a significant >>> improvement from this patch, and we can discuss the best steps to >>> address that. >> In short: I have a project.el backend for a large monorepo which has >> a >> project-files backend which returns only the subset of files which are >> relevant to work happening in a given clone. (Generally a user will >> have many clones and be doing different work in each one.) The >> relevant-files subset is determined by integration with the build >> system. >> So running find returns a vast number of files and then searches >> over >> those, whereas running a search over project-files searches a much >> smaller number of files. > > Neat. > >> Regarding your medium-term plans to improve project-files performance - >> wildly guessing, but perhaps you have in mind a way to run a subprocess >> that outputs the project-files list? Let's call it >> "project-files-process". And then project-files-process could be piped >> to grep instead, for maximum efficiency? If that was the idea, then my >> own backend could certainly have a project-files-process implementation >> too, for maximum efficiency. > > That might be step number 3, although I'm not sure yet which kind of > code will be required for the piping to be done efficiently enough. > > The other two things I was looking at are: > > - Use relative file names (less text to parse, memory to allocate, GC > to thrash). The awkward part is how to merge that with the idea that > project-files can include files from directories ("external > roots"). Split those off into a different method? Treat them as > separate projects to flat-map the lists of files at? > > - Add arguments to allow filtering the files using the underlying > tool. That can also result is much fewer files to parse in the > output under suitable circumstances (e.g. we'd be able to pass a > list of globs here). > > There is one implementation of the second item in the branch > scratch/etags-regen. > > And both items need to be done carefully enough to maintain some > backward compatibility. > > So unless you're in a hurry, give me a few weeks to get around to this. > > Further suggestions and patches are welcome, of course. I'm in no hurry. I will probably add this backend locally at my site in the meantime. We have no existing (non-trivial) xref-find-references backend, so speeding this one up isn't too urgent (it's not competing with anything), but definitely I am interested in project-files (and project.el in general) speed improvements and will try to help out as it becomes relevant.