From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Martin Pool" Newsgroups: gmane.emacs.devel Subject: Re: Emacs Bazaar repository Date: Fri, 14 Mar 2008 18:03:39 +1100 Message-ID: References: <87skyvse7k.fsf@xmission.com> <87fxuvl0gi.fsf@offby1.atm01.sea.blarg.net> <87bq5j19ok.fsf@workhorse.earlhome> <873aqtuasq.fsf@uwakimon.sk.tsukuba.ac.jp> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1205495949 14926 80.91.229.12 (14 Mar 2008 11:59:09 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 14 Mar 2008 11:59:09 +0000 (UTC) Cc: Jonathan Lange , Jason Earl , emacs-devel@gnu.org To: "Stephen J. Turnbull" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Mar 14 12:59:36 2008 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1Ja8ZM-0000T4-S5 for ged-emacs-devel@m.gmane.org; Fri, 14 Mar 2008 12:59:29 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ja8Yn-0001HK-MW for ged-emacs-devel@m.gmane.org; Fri, 14 Mar 2008 07:58:53 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Ja3xB-0003ci-J5 for emacs-devel@gnu.org; Fri, 14 Mar 2008 03:03:45 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Ja3x9-0003YZ-MN for emacs-devel@gnu.org; Fri, 14 Mar 2008 03:03:44 -0400 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ja3x9-0003YB-DK for emacs-devel@gnu.org; Fri, 14 Mar 2008 03:03:43 -0400 Original-Received: from mx20.gnu.org ([199.232.41.8]) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1Ja3x8-0004ca-Ns for emacs-devel@gnu.org; Fri, 14 Mar 2008 03:03:42 -0400 Original-Received: from wf-out-1314.google.com ([209.85.200.173]) by mx20.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1Ja3x8-0005m4-1d for emacs-devel@gnu.org; Fri, 14 Mar 2008 03:03:42 -0400 Original-Received: by wf-out-1314.google.com with SMTP id 29so3803431wff.24 for ; Fri, 14 Mar 2008 00:03:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=Jf/p25W6mT9tya3r8exe7W0Kh2YyXS8GVw3Dk2hJajI=; b=Fxyp/AtxBpvqUcFGIXACt8M4wszUoGYa3XQJl+NxnxUkxnl89B8frXCJ2kmqJS9aHxtqxFeBZjPzml04Z7mu2E9UdIrp5y7nnpSvn5RyFD/HLO9Ok/jf1zmdIJHtmh6UIROhqywO9LyRgi+uOcBv1FelJCEDG3dbuHCuTQIjLmM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=mtM1y1K69YhR9kXnfR+gPwkJau0VISoTMtRAmZo1/pand1fnqDXV0hCBaioz4/kYY6WqKUkosRHUVZsezFHV9gYjS+4fnP0WWYlJujbyBRNLIbV9uhgKnMxL/6xKui2QhmUR4lQvfn72sKGB5tN1Hi6VPxoxfwD+QsVWkr8RP/w= Original-Received: by 10.142.185.13 with SMTP id i13mr4748711wff.213.1205478219361; Fri, 14 Mar 2008 00:03:39 -0700 (PDT) Original-Received: by 10.142.157.3 with HTTP; Fri, 14 Mar 2008 00:03:39 -0700 (PDT) In-Reply-To: <873aqtuasq.fsf@uwakimon.sk.tsukuba.ac.jp> Content-Disposition: inline X-Google-Sender-Auth: f995e54e0109528a X-detected-kernel: by mx20.gnu.org: Linux 2.6 (newer, 2) X-detected-kernel: by monty-python.gnu.org: Linux 2.6, seldom 2.4 (older, 4) X-Mailman-Approved-At: Fri, 14 Mar 2008 07:58:09 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:92531 Archived-At: On 14/03/2008, Stephen J. Turnbull wrote: > Martin Pool writes: > > > I believe the main problem is that we are processing the whole graph > > to work out which revisions merged which others. > > What does this mean? That you can avoid pulling patches that cause > conflicts if they've already been applied? But that's not terribly > interesting in most logging or mass download applications. Bazaar shows revisions in a nested form like this: a b c d e f .. This means that c and d were merged into this branch in revision b, and were not previously present in the repository. This layout gives a good overview of the history, in our opinion a much better one than what you get from tools that just print one path through the graph or that print the revisions in arbitrary linear order. To calculate which revisions are newly merged is currently done by examining the whole graph. We will do two things to speed it up: faster loading of the graph from the repository, and caching some data in the branch. But possibly you don't want to know about merged in revisions, and that can be done with bzr log --short or --line. These should be able to do much less work, just walking down through the revision to print. mwh posted a patch for this, and we can take it further. > While I quoted Hoare on the root of all evil elsewhere, and continue > to take that as my default position, these numbers are uncomfortable. > I don't care how cold the CPU cache is, I don't want an attempt to > access a non-existent local repo to take 10s. Nor should a > disk-to-disk copy of a 400MB directory tree take 23.5m. The partial > clone times are heartening, although it's hard to guess what they > mean in terms of throughput. I want to point out that bzr branch locally, not in a single repository, is doing more than just copying all the files - it is filtering out and verifying the data relevant to the particular branch you're copying. If you just want to copy the data, use cp; if you want a new branch without copying, use init-repo to create a common repository directory above them. That said, it would probably be useful to have an option which copies the whole thing with no or fewer checks, and to speed up the way we do copy it. > # Initial one-branch clone. > $ time bzr clone http://bzr.notengoamigos.org/emacs/trunk/ earl-bzr > Branched 87515 revision(s). > > real 101m35.490s user 28m16.698s sys 2m7.827s > > # Clone from a local standalone branch into a shared repo. > $ time bzr clone ../earl-bzr trunk > Branched 87515 revision(s). > > real 23m27.626s user 13m57.869s sys 1m12.516s > > # Command line typo on a local URL, bzr figures it out in 10s. > $ time bzr branch EMACS_22_BASE > bzr: ERROR: Not a branch: "/Users/steve/Software/Emacs/emacs-bzr/EMACS_22_BASE/". > > real 0m10.183s user 0m0.457s sys 0m1.361s Maybe you could send (perhaps just on the bzr list) a run of that with "bzr --lsprof branch ...". It may be that the low cpu usage is caused by many things being pushed out of ram by building the emacs working tree in the previous command. > # Command line typo on a remote URL, bzr figures it out in 3.5s. > $ time bzr clone http://bzr.notengoamigos.org/emacs/EMACS_22_BASE > bzr: ERROR: Not a branch: "http://bzr.notengoamigos.org/emacs/EMACS_22_BASE/". > > real 0m3.427s user 0m0.499s sys 0m1.142s For me this remote urls takes about .6s; I may be closer to that server. At any rate it is only sending one http request which is the least that can be reasonably expected. :) -- Martin