Re: Emacs Bazaar repository

["Martin Pool" <mbp@sourcefrog.net>]

On 14/03/2008, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Martin Pool writes:
>
>   > I believe the main problem is that we are processing the whole graph
>   > to work out which revisions merged which others.
>
> What does this mean?  That you can avoid pulling patches that cause
>  conflicts if they've already been applied?  But that's not terribly
>  interesting in most logging or mass download applications.

Bazaar shows revisions in a nested form like this:

  a
  b
    c
    d
  e
  f
  ..

This means that c and d were merged into this branch in revision b,
and were not previously present in the repository.  This layout gives
a good overview of the history, in our opinion a much better one than
what you get from tools that just print one path through the graph or
that print the revisions in arbitrary linear order.

To calculate which revisions are newly merged is currently done by
examining the whole graph.  We will do two things to speed it up:
faster loading of the graph from the repository, and caching some data
in the branch.

But possibly you don't want to know about merged in revisions, and
that can be done with bzr log --short or --line.  These should be able
to do much less work, just walking down through the revision to print.
mwh posted a patch for this, and we can take it further.

>  While I quoted Hoare on the root of all evil elsewhere, and continue
>  to take that as my default position, these numbers are uncomfortable.
>  I don't care how cold the CPU cache is, I don't want an attempt to
>  access a non-existent local repo to take 10s.  Nor should a
>  disk-to-disk copy of a 400MB directory tree take 23.5m.  The partial
>  clone times are heartening, although it's hard to guess what they
>  mean in terms of throughput.

I want to point out that bzr branch locally, not in a single
repository, is doing more than just copying all the files - it is
filtering out and verifying the data relevant to the particular branch
you're copying.  If you just want to copy the data, use cp; if you
want a new branch without copying, use init-repo to create a common
repository directory above them.

That said, it would probably be useful to have an option which copies
the whole thing with no or fewer checks, and to speed up the way we do
copy it.

>  # Initial one-branch clone.
>  $ time bzr clone http://bzr.notengoamigos.org/emacs/trunk/ earl-bzr
>  Branched 87515 revision(s).
>
>  real    101m35.490s     user    28m16.698s      sys     2m7.827s
>
>  # Clone from a local standalone branch into a shared repo.
>  $ time bzr clone ../earl-bzr trunk
>  Branched 87515 revision(s).
>
>  real    23m27.626s      user    13m57.869s      sys     1m12.516s
>
>  # Command line typo on a local URL, bzr figures it out in 10s.
>  $ time bzr branch EMACS_22_BASE
>  bzr: ERROR: Not a branch: "/Users/steve/Software/Emacs/emacs-bzr/EMACS_22_BASE/".
>
>  real    0m10.183s       user    0m0.457s        sys     0m1.361s

Maybe you could send (perhaps just on the bzr list) a run of that with
"bzr --lsprof branch ...".  It may be that the low cpu usage is caused
by many things being pushed out of ram by building the emacs working
tree in the previous command.

>  # Command line typo on a remote URL, bzr figures it out in 3.5s.
>  $ time bzr clone http://bzr.notengoamigos.org/emacs/EMACS_22_BASE
>  bzr: ERROR: Not a branch: "http://bzr.notengoamigos.org/emacs/EMACS_22_BASE/".
>
>  real    0m3.427s        user    0m0.499s        sys     0m1.142s

For me this remote urls takes about .6s; I may be closer to that
server.  At any rate it is only sending one http request which is the
least that can be reasonably expected. :)

-- 
Martin