From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#62837: [PATCH] Add a semantic-symref backend which uses xref-matches-in-files Date: Wed, 19 Apr 2023 04:10:24 +0300 Message-ID: <37ee089e-5c42-15c6-c8c1-48642bf4e180@yandex.ru> References: <5e6eddd5-4b38-5765-05f3-dd6c1927edd3@yandex.ru> <871qkkn720.fsf@catern.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="30966"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Cc: Spencer Baugh , 62837@debbugs.gnu.org To: sbaugh@catern.com Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Apr 19 03:11:20 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1powLw-0007s9-7B for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 19 Apr 2023 03:11:20 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1powLg-0005kt-05; Tue, 18 Apr 2023 21:11:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1powLe-0005kd-4s for bug-gnu-emacs@gnu.org; Tue, 18 Apr 2023 21:11:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1powLd-0004R2-Tf for bug-gnu-emacs@gnu.org; Tue, 18 Apr 2023 21:11:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1powLd-0006Y0-PO for bug-gnu-emacs@gnu.org; Tue, 18 Apr 2023 21:11:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 19 Apr 2023 01:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62837 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 62837-submit@debbugs.gnu.org id=B62837.168186663825116 (code B ref 62837); Wed, 19 Apr 2023 01:11:01 +0000 Original-Received: (at 62837) by debbugs.gnu.org; 19 Apr 2023 01:10:38 +0000 Original-Received: from localhost ([127.0.0.1]:60727 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1powLF-0006X2-Le for submit@debbugs.gnu.org; Tue, 18 Apr 2023 21:10:38 -0400 Original-Received: from forward501c.mail.yandex.net ([178.154.239.209]:59462) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1powLA-0006Wl-EC for 62837@debbugs.gnu.org; Tue, 18 Apr 2023 21:10:35 -0400 Original-Received: from mail-nwsmtp-smtp-production-canary-88.sas.yp-c.yandex.net (mail-nwsmtp-smtp-production-canary-88.sas.yp-c.yandex.net [IPv6:2a02:6b8:c08:1d21:0:640:c9e4:0]) by forward501c.mail.yandex.net (Yandex) with ESMTP id 409745F1B4; Wed, 19 Apr 2023 04:10:30 +0300 (MSK) Original-Received: by mail-nwsmtp-smtp-production-canary-88.sas.yp-c.yandex.net (smtp/Yandex) with ESMTPSA id RAR4N3HDdqM0-yBW1moyX; Wed, 19 Apr 2023 04:10:29 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1681866629; bh=uw0KeozqTZ3gtcmDDKO3ttxfDy8f+sDeQtgmPNgA8XM=; h=In-Reply-To:From:Subject:Message-ID:Cc:References:Date:To; b=hCjHsYPonpqkHcoS/sdwJf8Q6DExncQcCaAsBi7mx38M8EXz3pKGFh0Ao/JBSj2ot Q3nYjahvjEeq/lc9Gcy58h3FWa6OLJ04UJ1D6oE6xjdwziZXz3gU9axHE41sgLpimE 0l8e/Ote3wH+DoMYu4/sgrQJKOKI/jdefnM0Kn4M= Authentication-Results: mail-nwsmtp-smtp-production-canary-88.sas.yp-c.yandex.net; dkim=pass header.i=@yandex.ru Original-Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailauth.nyi.internal (Postfix) with ESMTP id 6333027C005B; Tue, 18 Apr 2023 21:10:27 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 18 Apr 2023 21:10:27 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdelledggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegughhuthhovheshigrnhguvgigrdhruheqnecuggftrfgrth htvghrnheptdffgeegkeelteevtdekleethfeftdduvdegkedtkedujefhfedtveeftdff udevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hguhhtohhvodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqddufeeffeelleeh hedvqddvleegjeejjeejiedqughguhhtohhvpeephigrnhguvgigrdhruhesfhgrshhtmh grihhlrdgtohhm X-ME-Proxy: Feedback-ID: ib1d9465d:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 18 Apr 2023 21:10:26 -0400 (EDT) Content-Language: en-US In-Reply-To: <871qkkn720.fsf@catern.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:260251 Archived-At: On 16/04/2023 00:56, sbaugh@catern.com wrote: >> Perhaps you could describe your case where you *did* see a significant >> improvement from this patch, and we can discuss the best steps to >> address that. > > In short: I have a project.el backend for a large monorepo which has a > project-files backend which returns only the subset of files which are > relevant to work happening in a given clone. (Generally a user will > have many clones and be doing different work in each one.) The > relevant-files subset is determined by integration with the build > system. > > So running find returns a vast number of files and then searches over > those, whereas running a search over project-files searches a much > smaller number of files. Neat. > Regarding your medium-term plans to improve project-files performance - > wildly guessing, but perhaps you have in mind a way to run a subprocess > that outputs the project-files list? Let's call it > "project-files-process". And then project-files-process could be piped > to grep instead, for maximum efficiency? If that was the idea, then my > own backend could certainly have a project-files-process implementation > too, for maximum efficiency. That might be step number 3, although I'm not sure yet which kind of code will be required for the piping to be done efficiently enough. The other two things I was looking at are: - Use relative file names (less text to parse, memory to allocate, GC to thrash). The awkward part is how to merge that with the idea that project-files can include files from directories ("external roots"). Split those off into a different method? Treat them as separate projects to flat-map the lists of files at? - Add arguments to allow filtering the files using the underlying tool. That can also result is much fewer files to parse in the output under suitable circumstances (e.g. we'd be able to pass a list of globs here). There is one implementation of the second item in the branch scratch/etags-regen. And both items need to be done carefully enough to maintain some backward compatibility. So unless you're in a hurry, give me a few weeks to get around to this. Further suggestions and patches are welcome, of course.