From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Date: Thu, 20 Jul 2023 21:54:32 +0300 Message-ID: <3a33eb87-4f1c-efee-3d2d-d3ae2c3e7e4b@gutov.dev> References: <837cqv41ob.fsf@gnu.org> <87mszqixhh.fsf@catern.com> <4d4f029f-f32f-13df-ffc3-3952d62d8bb3@gutov.dev> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="29991"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: sbaugh@catern.com, Eli Zaretskii , 64735@debbugs.gnu.org To: Spencer Baugh Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Jul 20 20:55:23 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qMYo7-0007c6-7i for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 20 Jul 2023 20:55:23 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qMYns-0004P2-HH; Thu, 20 Jul 2023 14:55:09 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMYnq-0004Oa-0b for bug-gnu-emacs@gnu.org; Thu, 20 Jul 2023 14:55:06 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qMYnn-00059d-BH for bug-gnu-emacs@gnu.org; Thu, 20 Jul 2023 14:55:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qMYnm-0005TD-I3 for bug-gnu-emacs@gnu.org; Thu, 20 Jul 2023 14:55:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 20 Jul 2023 18:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64735 X-GNU-PR-Package: emacs Original-Received: via spool by 64735-submit@debbugs.gnu.org id=B64735.168987928721000 (code B ref 64735); Thu, 20 Jul 2023 18:55:02 +0000 Original-Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:54:47 +0000 Original-Received: from localhost ([127.0.0.1]:59891 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMYnX-0005Se-7L for submit@debbugs.gnu.org; Thu, 20 Jul 2023 14:54:47 -0400 Original-Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:42967) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMYnT-0005SO-CO for 64735@debbugs.gnu.org; Thu, 20 Jul 2023 14:54:46 -0400 Original-Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id C871C3202AEB; Thu, 20 Jul 2023 14:54:36 -0400 (EDT) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 20 Jul 2023 14:54:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689879276; x=1689965676; bh=jmu7/QfDNUt5SDBOBPFWjQRSk/qub2wO593 n2iePy0E=; b=cf6XbX662hXkX442wPgZ1cx4N7CgTaV42f2E0C7uDrEODKm0svp CHvlI28ED+wftitd+JFue49DQyIuCwxStee4PFkgU1N9QxP/Hn5iLkSNURelPhzJ /U3tvBm+CSf2b908wmdr0J3j9Dacf54OikLjyVX257n+ZeNPBfasIrKVJSMn/ih1 wsYjd6WEfcM8JYc7wWdGZT5xqnGxKEH3e/51XBUEK5LgLLRBYoAN4igJSmrlvsfP 320qz368nhNcvJlG7y2EL9G6YjfofcwcLMeuHct4MZs1jRXjrzuW+veXesr5wpyi pL6Gj8E1qlK7OSRGIYX9bU+qRUTaBskTbEA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689879276; x=1689965676; bh=jmu7/QfDNUt5SDBOBPFWjQRSk/qub2wO593 n2iePy0E=; b=NXyEf2CKwbKnGIHmKhlqQxy+v2O4CqWBK1LfjAXNWJW36uMhGHf 6KTIStziOoBq37qK2wYLVpkj3/IDZIS7SyEIKs0uIo6WWn8I487O3jXqsuq9QVbi qY2Zca8kTlwiVa8/s/kHSClP0Ijnxsxj2IBTCf1p34Gk55UAP6NxLKlh6SFrDLoE sUNnGKuAwyiTG7ZZR5XSMYiAPVACg7gMAumRqCSXR3j01adUt5wKaNq1WT0Hscuk TeDJc0prnb9w1ZM/pSryjYa9c2B/LUba323FEeATvF6MD/h2pUk8scue+UJpmWKf 1fko45n6+Z7KCeRtmRVPVspZ9gTVmudwqKA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 14:54:34 -0400 (EDT) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:265631 Archived-At: On 20/07/2023 16:43, Spencer Baugh wrote: >> That's only a problem when the default file listing logic is used (and >> we usually delegate to something like 'git ls-files' instead, when the >> vc-aware backend is used). > > Hm, yes, but things like C-u project-find-regexp will use the default > find-based file listing logic instead of git ls-files, as do a few other > things. Right. > I wonder, could we just go ahead and make a vc function which is > list-files(GLOBS) and returns a list of files? Both git and hg support > this. Then we could have C-u project-find-regexp use that instead of > find, by taking the cross product of dirs-to-search and > file-name-patterns-to-search. (And this would let me delete a big chunk > of my own project backend, so I'd be happy to implement it.) I started out on this inside the branch scratch/project-regen. Didn't have time to dedicate to it recently, but the basics are there, take a look (the method is called project-files-filtered). The difficulty with making such changes, is the project protocol grows in size, it becomes difficult for a user to understand what is mandatory, what's obsolete, and how to use it, especially in the face of backward compatibility requirements. Take a look, feedback is welcome, it should help move this forward. We should also transition to returning relative file names when possible, for performance (optionally or always). > Fundamentally it seems a little silly for project-ignores to ever be > used for a vc project; if the vcs gives us ignores, we can probably just > ask the vcs to list the files too, and it will have an efficient > implementation of that. Possibly, yes. But there will likely remain cases when the project-files could stay useful for callers, to construct some bigger command line for some new feature. Though perhaps we'll be able to drop that need by extracting the theoretically best performance from project-files (using a process object or some abstraction), to facilitate low-overhead piping. > If we do that uniformly, then this find slowness would only affect > transient projects, and transient projects pull their ignores from > grep-find-ignored-files just like rgrep, so improvements will more > easily be applied to both. (And maybe we could even get rid of > project-ignores entirely, then?) Regarding removing it, see above. And it'll take a number of years anyway ;-(