From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: vc-dir operation is very slow on large git repositories in Emacs 26.1 Date: Fri, 22 Jun 2018 11:57:18 +0300 Message-ID: <83efgzqjv5.fsf@gnu.org> References: <83k1qtsbgi.fsf@gnu.org> <83zhzoqkgv.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1529657892 10418 195.159.176.226 (22 Jun 2018 08:58:12 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 22 Jun 2018 08:58:12 +0000 (UTC) Cc: emacs-devel@gnu.org To: Alex Harsanyi Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Jun 22 10:58:08 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fWHtQ-0002cC-GC for ged-emacs-devel@m.gmane.org; Fri, 22 Jun 2018 10:58:08 +0200 Original-Received: from localhost ([::1]:60177 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fWHvX-0007xo-TX for ged-emacs-devel@m.gmane.org; Fri, 22 Jun 2018 05:00:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54152) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fWHsb-0006JW-4y for emacs-devel@gnu.org; Fri, 22 Jun 2018 04:57:18 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fWHsW-0006cX-6R for emacs-devel@gnu.org; Fri, 22 Jun 2018 04:57:17 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35882) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fWHsW-0006cP-2D; Fri, 22 Jun 2018 04:57:12 -0400 Original-Received: from [176.228.60.248] (port=3552 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fWHsU-0007hW-Sv; Fri, 22 Jun 2018 04:57:11 -0400 In-reply-to: (message from Alex Harsanyi on Fri, 22 Jun 2018 09:12:29 +0800) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:226585 Archived-At: > From: Alex Harsanyi > Date: Fri, 22 Jun 2018 09:12:29 +0800 > Cc: emacs-devel@gnu.org > > I set `vc-command-messages`, which prints out the subprocesses run by > `vc-do-command` and also updated `vc-do-command` to show the time it took for > each process. It turns out that Emacs 26.1 does run one extra command: > > In Emacs 25.2, a "vc-dir" command runs the following: > > Running git --no-pager update-index --refresh . in background... done > 0.48 seconds > Running git --no-pager diff-index --relative -z -M HEAD -- . in > background... done 0.42 seconds > Running git --no-pager ls-files -z -o --directory --no-empty-directory > --exclude-standard -- . in background... done 0.10 seconds > > In Emacs 26.2, a "vc-dir" command runs the following: > > Running git --no-pager update-index --refresh . in background... done > 0.46 seconds > Running git --no-pager diff-index --relative -z -M HEAD -- . in > background... done 0.46 seconds > Running git --no-pager ls-files -z -c -s -- . in background... done 4.49 seconds > Running git --no-pager ls-files -z -o --directory --no-empty-directory > --exclude-standard -- . in background... done 0.09 seconds > > The extra command is "git --no-pager ls-files -z -c -s -- ." and it takes 4.5 > seconds to run (I ran each test multiple times and the times are consistent). > However, when I run that command in a terminal, it finishes in less than a > second. But the times you cited originally were much longer: 2 min. What caused such a stark difference, from 2 min to just 4.5 sec? > An Emacs-Lisp program to run this script, but ignore all output, it takes 16 > seconds, in both 25.2 and 26.1: Once again, 16 sec is much shorter than 2 min you originally cited. > It seems reading output from a process in Emacs is somehow limited to 64 Kb / > second -- I verified this with procmon. It looks like the behavior is the > same in 25.2 and 26.1, it is just that in vc-dir 25.2 does not run a git > command that produces a lot of output. Does it help to set w32-pipe-buffer-size to some large value, like 16384, and/or decrease w32-pipe-read-delay? Thanks.