From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Sean Allred Newsgroups: gmane.emacs.bugs Subject: bug#73320: [PATCH] project--vc-list-files: use Git's sparse-index Date: Tue, 17 Sep 2024 11:55:43 -0500 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28945"; mail-complaints-to="usenet@ciao.gmane.io" To: 73320@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Sep 17 18:57:05 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sqbVg-0007JH-UD for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 17 Sep 2024 18:57:05 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sqbVU-0006sy-9Z; Tue, 17 Sep 2024 12:56:52 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sqbVQ-0006si-AY for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 12:56:48 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sqbVQ-00023Z-1q for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 12:56:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:Subject; bh=JOYOH4yhCF5Ms2cqAGuuWfD+M2oDCacc+iudk+KZxiw=; b=oKf9TcENNk7sCLvsWBkYExN88fYE7xbmRBDYcGY0nkTgOWirG7z6eV6IUNRdDX0cw+Ivlc3CQCwY/uiIrIzy9fFRfqUV0karn49/WPnQllXbaLQm5MZ/Lk3neRtZwvEBQcy398qACUdfNDyUzkhHY7nAdFes2zL5jk2THA1dWxUZ9avhP1gQ6xXUUXs4s0DdKltnyZJDbVnVOiIIUCMhxb3sIn6dUxc4YPZR6lsZynrX45Ex3kd0LaSHFlgyBU9nQVUWe6ZNxDGtePlDKIOKbPthoq+hGRVy4zhI0oJekWXOFFLquG9flT0+DwyAtVuwAYcCehUK6KSgq8CqToys5w==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sqbVe-00088V-L4 for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 12:57:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Sean Allred Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 17 Sep 2024 16:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 73320 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.172659217431193 (code B ref -1); Tue, 17 Sep 2024 16:57:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 17 Sep 2024 16:56:14 +0000 Original-Received: from localhost ([127.0.0.1]:55830 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sqbUr-000872-Ru for submit@debbugs.gnu.org; Tue, 17 Sep 2024 12:56:14 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:41138) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sqbUp-00086u-Vh for submit@debbugs.gnu.org; Tue, 17 Sep 2024 12:56:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sqbUZ-0006pt-4T for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 12:55:56 -0400 Original-Received: from mail-yb1-xb33.google.com ([2607:f8b0:4864:20::b33]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sqbUT-00021T-AI for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 12:55:52 -0400 Original-Received: by mail-yb1-xb33.google.com with SMTP id 3f1490d57ef6-e0b7efa1c1bso5820585276.3 for ; Tue, 17 Sep 2024 09:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726592147; x=1727196947; darn=gnu.org; h=mime-version:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=JOYOH4yhCF5Ms2cqAGuuWfD+M2oDCacc+iudk+KZxiw=; b=C93v5lGbe4yGUp1X2sPW2bo3yYMa9WwBjkhjHIQrbDtYo0T3NKxhCRBvZWC7r229P2 5CaIcLV1X88nstTfN9LcS/YYAJVpausJ6VRIWqdut2puMDt9KH7ExWSNwbyGnhrJl/vv uzP8KRdVzh/iMcXZ5OE4UTQ5yX/+r3n4kCWW2QTicKViWaFnVShqlcbDeYM8QRUz3p3J 0yXMg3LHRIZkEFEh+E28TxTPEZOnb5DA17bogUISeyHE5EYaMa0um6DS/zBdGjbkYohA 1tNfh8dIbgHH+P20QeIyX+Xs4b4V+N0mZtRmsS0LMhAqPG7ah3KwOPsf/uJyWB80TtJW lm6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726592147; x=1727196947; h=mime-version:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=JOYOH4yhCF5Ms2cqAGuuWfD+M2oDCacc+iudk+KZxiw=; b=q/84FYM+5tePUdmLKbDOCZClD2butmyBhqwo5tuv/fWfyFDT+sORvQCn/SL98sBDh/ KO5O4nP9NVxd71iYMu9lFYaYrvvdpQ0sPY8pLjGMm6obEDy7ashGLDok+aUGubUHRvwL rQo/qZC4oegFny+LgXHVA6BV3IQpz44Op1sV/6SGbAJLufsMjDkL948qdjxU52hd/SHs yQ5+KjpbhqSbjYW2OMjStjo8w5Rb8YxvfxIUA0SjeHMosOEfEPY2CGMzDgTO2ksz12HL LyqU2QYxQRhyJKjI7yyViG6+cZzZ6fuM2xTNxfW9SkCay+j/n1JFAM+cJd51faH63efs GSvA== X-Gm-Message-State: AOJu0Yzd+14+K0HWsk50efdbJWDYDEbbfoVHkrbECJCYYnCx9CXMVNGi 7iGcNHG8mxVqAx8He4UXXC+tSjw/F0X1aFf86pVyOYVFwJjDgtw7EQySMg== X-Google-Smtp-Source: AGHT+IGbmLYccGHqJbyKECKiRjR/ouIGC/IC8FAz4QLE82DT/AAjNrp5cnoKPzVjQXXa0J+SS7cBBw== X-Received: by 2002:a05:690c:640e:b0:6db:de99:28ae with SMTP id 00721157ae682-6dbde992a62mr107339937b3.17.1726592145948; Tue, 17 Sep 2024 09:55:45 -0700 (PDT) Original-Received: from epic96565.epic.com (pat-verona-k.epic.com. [199.204.56.216]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6dc314c7a9fsm13240557b3.115.2024.09.17.09.55.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2024 09:55:44 -0700 (PDT) Received-SPF: pass client-ip=2607:f8b0:4864:20::b33; envelope-from=allred.sean@gmail.com; helo=mail-yb1-xb33.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:291952 Archived-At: --=-=-= Content-Type: text/plain Tags: patch Hello! I noticed that `C-x p f` (M-x project-find-file) took an incredibly long time to run on our monorepo -- even when using a sparse index -- and I tracked the problem down to this function. Adding `--sparse` to the `git-ls-files` invocation resolves the issue handily, albeit with the quirk of still showing top-level directories that are excised from the sparse index (which we may want to remove from the return value of this function since it does say it returns /files/). I'm expecting at least one more version of this patch before it's even considered for merge. Given that this change removes many, many results from the return value of `project--vc-list-files', I suspect this would be a breaking change for some use case that I'm not considering. I'm hoping the list can provide some feedback on the best way to make this configurable. I've not really hacked in project.el before and am not super familiar with its internals. If you prefer, you may pull this patch from my fork on github at url: git@github.com:vermiculus/emacs.git branch: sa/sparse-index Keep in mind I may be updating this branch with feedback from this list as it comes in. -Sean In GNU Emacs 29.1 (build 1, aarch64-apple-darwin21.6.0, NS appkit-2113.60 Version 12.6.6 (Build 21G646)) of 2023-07-30 built on armbob.lan Windowing system distributor 'Apple', version 10.3.2487 System Description: macOS 14.6.1 Configured using: 'configure --with-ns '--enable-locallisppath=/Library/Application Support/Emacs/${version}/site-lisp:/Library/Application Support/Emacs/site-lisp' --with-modules 'CFLAGS=-DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT' --with-x-toolkit=no' --=-=-= Content-Type: text/patch Content-Disposition: attachment; filename=0001-project-vc-list-files-use-Git-s-sparse-index.patch >From e72c013afbafad3faf19f98acb8d0387fdf045ba Mon Sep 17 00:00:00 2001 From: Sean Allred Date: Tue, 17 Sep 2024 10:07:41 -0500 Subject: [PATCH] project--vc-list-files: use Git's sparse-index When dealing with exceptionally large Git repositories, the performance of `project-find-file` can suffer dramatically as the list of files is collected for completion. This adds insult to injury when you consider cases where the developer has configured the repository to use a sparse checkout where the vast majority of these files are not even present on disk and are not valid candidates for completion. Unconditionally pass along the `--sparse` option to `git-ls-files` to coerce `project--vc-list-files` only consider files present in the sparse index (and presumably on disk). No consideration is made to make this configurable as it is currently unclear as to the best way to do this -- whether to follow the example of the `include-untracked` symbol in this function or to adopt some other mechanism. --- lisp/progmodes/project.el | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el index b29d5ed5404..82cb79322dd 100644 --- a/lisp/progmodes/project.el +++ b/lisp/progmodes/project.el @@ -663,7 +663,7 @@ project--vc-list-files (pcase backend (`Git (let* ((default-directory (expand-file-name (file-name-as-directory dir))) - (args '("-z")) + (args '("-z", "--sparse")) (vc-git-use-literal-pathspecs nil) (include-untracked (project--value-in-dir 'project-vc-include-untracked base-commit: 7d365a2d72d8e656262205827cc5fdf423c3a41f -- 2.46.0 --=-=-= Content-Type: text/plain -- Sean Allred --=-=-=--