From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Sean Allred Newsgroups: gmane.emacs.bugs Subject: bug#73320: [PATCH] project--vc-list-files: use Git's sparse-index Date: Tue, 17 Sep 2024 19:36:40 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32708"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 73320@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Sep 18 02:39:02 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sqiik-0008MN-BE for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 18 Sep 2024 02:39:02 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sqiiW-0000Bz-Nj; Tue, 17 Sep 2024 20:38:48 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sqiiV-0000Bc-87 for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 20:38:47 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sqiiU-0001FN-SD for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 20:38:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=4chl5jCAOVfwJTot+zD697tY36EkTrOJuwSG2w2aPPY=; b=GLDZFzZjR8mirbLBwA2TeFngMG1fofi8EXWwLAuXlHsMVV6r1uT7wTiYzlLM0E6MBMmfWfL7dWMTE5Yw80aKsTwqb9UKlZjO3LFt12mkWI6SOfB9afhkbGDx3n9kpC4hNI9pp2ZWNDg8VDH/MhC4IWlryrXlq70VYzX9aVZoTiI+I7KqKi8PwdU3ndfSHBb5sNG913LEDouEWtKgmsu2ZKnJTpXdWh/EaT0FxqvwBNm9fTZTd6+jxdbnHafxeOgfDbUrHSg/EPSH9XTc7gi+9+JL1DsiAcoooKo8C8L5Qz+xJe8es7eWJIapcFpU6Epj7uAKaD3baPrGnt3X/bzZug==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sqiik-0000Gx-56 for bug-gnu-emacs@gnu.org; Tue, 17 Sep 2024 20:39:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Sean Allred Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 18 Sep 2024 00:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 73320 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 73320-submit@debbugs.gnu.org id=B73320.1726619885944 (code B ref 73320); Wed, 18 Sep 2024 00:39:02 +0000 Original-Received: (at 73320) by debbugs.gnu.org; 18 Sep 2024 00:38:05 +0000 Original-Received: from localhost ([127.0.0.1]:56257 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sqihp-0000F9-5b for submit@debbugs.gnu.org; Tue, 17 Sep 2024 20:38:05 -0400 Original-Received: from mail-qv1-f45.google.com ([209.85.219.45]:61591) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sqihn-0000EQ-EM for 73320@debbugs.gnu.org; Tue, 17 Sep 2024 20:38:03 -0400 Original-Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-6c548eb3354so49502276d6.2 for <73320@debbugs.gnu.org>; Tue, 17 Sep 2024 17:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726619802; x=1727224602; darn=debbugs.gnu.org; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=4chl5jCAOVfwJTot+zD697tY36EkTrOJuwSG2w2aPPY=; b=hhG1/GIHUK2ku83J3WMyCL04yKORmifeKe3IXaCU6YwHpKB93RhsEpbYUyPib0WS2u z/mC5u4iGVgVFXqhrKlrCdZ2Fuh3T23XsItwEotMtz4FhptVSMTUy7hlliIR8eZcDsg9 L/t0fjFh+4zOoTJe2ZZuogjNIjaNAiBTUSYp+mPhrjHNLKH4OvzQaFkJ60oXjVqLFFGD UDeC3G/HFD2YvoipIUWbrLXr7WCriZj/33BOCz7532Cm2yhhAvsY6ynuahFEjmGA5aUo 6V/eUu/QXv/i/xc92bxAEhWEA5F7D/w70B1CYUUMIcJvn+Hz5klMV5AMKEtsGKBrlMs6 ol0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726619802; x=1727224602; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4chl5jCAOVfwJTot+zD697tY36EkTrOJuwSG2w2aPPY=; b=pyJOqdtTdd+7YnI0dOeO0gEzg8jqMDtPbjbM80nHaKnojrhNyGAoXeheKMPTIlDyil w1zt9pymy4IiTGbLKikeJQuQhvLUUvA+ZA8Ancb4kbf6Xv7O17bE5dHC6TzN7kTx4Fe+ +vmR5GlP08I0HGztlO4ypVjJ0Ix96qvCCDt7NNvli0iIORJ6xYaTHLh10zEwG0nzYVjp ptnVOL/NMlU+3Sv3tgiZ4Cg1+GP4ftCsgHlBBK4yzJAV8Kki8bu/OhjPk1bm67LxnkA+ 0IQV/Wy6K1mY/ain7ZidAH+KE9TacOZ05SLvGr+gLCXzzhb3tv2ScKPnCJLxRylnyKB3 ac0Q== X-Gm-Message-State: AOJu0YycR3VPvoEoD04ovu06CJGCVaEO+DVcIEdY8kFI2+L4aqSkWtxG +rCUnVHSWD4Dm3b3BIAwZR8dloxGxeJFQNp4pnyfHiKKdTl8SCnq+ooVxw== X-Google-Smtp-Source: AGHT+IEqkOiAp5jHYde7FsbUf0pSa1i/1OBaa1I0/vS5LrMotSGZe4OwWwkF+WsN3lL9zZUxFTjBNg== X-Received: by 2002:a05:6214:4a06:b0:6c5:279b:5dc7 with SMTP id 6a1803df08f44-6c573568623mr337762486d6.25.1726619801917; Tue, 17 Sep 2024 17:36:41 -0700 (PDT) Original-Received: from epic96565.epic.com (syn-075-135-086-025.res.spectrum.com. [75.135.86.25]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6c58c635395sm39799016d6.52.2024.09.17.17.36.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2024 17:36:41 -0700 (PDT) In-Reply-To: (Dmitry Gutov's message of "Wed, 18 Sep 2024 01:54:17 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:291985 Archived-At: Dmitry Gutov writes: > The submitted patch has a comma inside which results in an error Well that's embarrassing. I've resolved this in my branch. I'm not sure why I wasn't seeing an issue locally; perhaps I made this error somehow after I eval'd the form for testing. Thanks for catching! > But let's start from the beginning. Could you help me set up a sparse > repo that would help test out the change? > > I took a large-ish checkout with shallow history and set up the sparse > config like this: > > git sparse-checkout init --cone > git sparse-checkout set gfx media > > With that, both 'git status' and 'ls' behave as expected - no extra > directories around (just the top-level files and two subdirs). > > But 'git ls-files' and 'git ls-files --sparse' continue to show the > same large output. Any idea what could be wrong? My version of Git, > perhaps? Which is 2.40.1. Sure! You're very close. The key is here in git-ls-files(1): --sparse If the index is sparse, show the sparse directories without expanding to the contained files. Sparse directories will be shown with a trailing slash, such as "x/" for a sparse directory "x". combined with this in git-sparse-checkout(1): Use the --[no-]sparse-index option to use a sparse index (the default is to not use it). I fell into this too when setting up my worktree for testing; normally our internal tooling takes care of this for me. To resolve this issue in an existing checkout, you can simply git sparse-checkout init --sparse-index and that will rewrite your index file using sparse-index rules. > Yeah, I expect project-find-regexp, project-search, > project-query-replace-regexp might start misbehaving without > additional filtering -- either throwing up errors or, best case, > continuing to search through the "hidden" directories. Not sure how best to track that we should come back to this, but yeah. It seems like the right place to add some sort of switch would be in the `project-files` defmethod. From here, it looks like all the functions you mention could choose the behavior right for them. (Based on the function names alone -- it seems they would /also/ be interested in operating on only those files which exist on disk.) Incidentally looking at the version check within `project-files`, it's worthwhile to point out that `--sparse` is likely /not/ compatible with ancient versions of Git. Does vc have any sort of policy on requiring recent versions of these tools? If the answer is 'not really', I'll additionally want to add some sort of protection against using `--sparse` when the Git version won't understand it. This should be easy enough to do within the implementation of `project--vc-list-files`. -- Sean Allred