From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Tom Tromey Newsgroups: gmane.emacs.devel Subject: Re: Hideously slow VC status queries fixed Date: Sat, 29 Dec 2007 12:18:40 -0700 Message-ID: References: <20071227001113.3EDFE830B6E@snark.thyrsus.com> <20071227021940.GA17434@thyrsus.com> Reply-To: Tom Tromey NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1198957704 17921 80.91.229.12 (29 Dec 2007 19:48:24 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 29 Dec 2007 19:48:24 +0000 (UTC) Cc: emacs-devel@gnu.org To: esr@thyrsus.com Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Dec 29 20:48:37 2007 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1J8hfa-0006zl-0p for ged-emacs-devel@m.gmane.org; Sat, 29 Dec 2007 20:48:30 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1J8hfE-0004qN-Jq for ged-emacs-devel@m.gmane.org; Sat, 29 Dec 2007 14:48:08 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1J8hf9-0004pH-UC for emacs-devel@gnu.org; Sat, 29 Dec 2007 14:48:03 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1J8hf5-0004nY-6T for emacs-devel@gnu.org; Sat, 29 Dec 2007 14:48:03 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1J8hf4-0004nR-UD for emacs-devel@gnu.org; Sat, 29 Dec 2007 14:47:58 -0500 Original-Received: from mx1.redhat.com ([66.187.233.31]) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1J8hf4-00011u-HU for emacs-devel@gnu.org; Sat, 29 Dec 2007 14:47:58 -0500 Original-Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id lBTJlqvq030976; Sat, 29 Dec 2007 14:47:52 -0500 Original-Received: from pobox.corp.redhat.com (pobox.corp.redhat.com [10.11.255.20]) by int-mx1.corp.redhat.com (8.13.1/8.13.1) with ESMTP id lBTJlq3H025032; Sat, 29 Dec 2007 14:47:52 -0500 Original-Received: from opsy.redhat.com (ton.yyz.redhat.com [10.15.16.15]) by pobox.corp.redhat.com (8.13.1/8.13.1) with ESMTP id lBTJlpbM002401; Sat, 29 Dec 2007 14:47:52 -0500 Original-Received: by opsy.redhat.com (Postfix, from userid 500) id 778C1C8803B; Sat, 29 Dec 2007 12:18:41 -0700 (MST) X-Attribution: Tom In-Reply-To: <20071227021940.GA17434@thyrsus.com> (Eric S. Raymond's message of "Wed\, 26 Dec 2007 21\:19\:40 -0500") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.990 (gnu/linux) X-Scanned-By: MIMEDefang 2.58 on 172.16.52.254 X-detected-kernel: by monty-python.gnu.org: Linux 2.6 (newer, 3) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:85635 Archived-At: >>>>> "Eric" == Eric S Raymond writes: Eric> I don't think the bottlneck is VC anymore. In the normal cases (CVS, Eric> SVN, Mercurial), VC now generates just one (1) command. I suppose Eric> parsing time could be an issue -- has anyone profiled 26,499 regexp Eric> matches lately ? I looked at this a little bit. Here's the top of the elp output (any function after this took less than 1 second elapsed time). I used elp-instrument-package with "vc-" as the argument (I never used elp before; let me know if I did something wrong). Function Name Call Count Elapsed Time Average Time =================================== ========== ============ ============ vc-call-backend 346 31.258875999 0.0903435722 vc-backend 25650 15.846656999 0.0006178033 vc-registered 34 8.9951210000 0.2645623823 vc-stay-local-p 1 7.837376 7.837376 vc-file-getprop 51331 7.2145619999 0.0001405498 vc-file-setprop 238371 6.6009059999 2.769...e-05 vc-state 25613 2.4212969999 9.453...e-05 Interestingly, vc-svn doesn't show up here at all. I also looked at this with oprofile. This is interesting too: samples % image name symbol name 1695163 48.2143 emacs Fbyte_code 295864 8.4150 emacs arithcompare 238998 6.7976 emacs mark_object Regular expression matching comes in at #13: 31889 0.9070 emacs re_match_2_internal What this says to me is that we're simply running a lot of plain emacs lisp, and that regex matching is not a big problem here. (I can send the whole list if you are interested.) I wonder if there is a way to do a lot less work. For instance, could we have VC look only at files that are not 'up-to-date? In my tree this would mean processing 24 files -- 3 orders of magnitude fewer. I think this would be a pretty common result for large trees, since it is rare to have a patch that touches a substantial fraction of gcc. Thank you for working on this. I appreciate it quite a bit. Tom