Emacs Bazaar repository

Emacs Bazaar repository

[Jason Earl]

After a few false starts I have been successful (at least as far as I
can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
is interested in playing with the results you can check out the HEAD of
the CVS repository with:

bzr clone http://bzr.notengoamigos.org/emacs/trunk/

The other branches are available as well and have URLs like:

http://bzr.notengoamigos.org/emacs/branches/<branch-name>/

For more information please see:

http://bzr.notengoamigos.org/

This repository is obviously something that should mostly interest Emacs
developers, but I'm sending this email to the bazaar mailing list as
well in case they are curious.

Jason

Re: Emacs Bazaar repository

[Stefan Monnier]

> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:

> bzr clone http://bzr.notengoamigos.org/emacs/trunk/

> The other branches are available as well and have URLs like:

> http://bzr.notengoamigos.org/emacs/branches/<branch-name>/

> For more information please see:

> http://bzr.notengoamigos.org/

> This repository is obviously something that should mostly interest Emacs
> developers, but I'm sending this email to the bazaar mailing list as
> well in case they are curious.

Thank you very much.  Now we'll be able to finally try out Bzr and see
if it's good enough.
I encourage everyone to try it out,


        Stefan

Re: Emacs Bazaar repository

[dhruva]

Well, now we have cvs, git, mercurial and bazaar. Having a subversion
repository will complete the long story.
I just hope this indecision will not continue for too long and confuse
people in terms of which repository to use. So far, i just had to
mention 'emacs head' has this problem on some platform in reporting
bugs. I have now started adding the repo info too!
I request the new maintainers to make a decision in the near future
and streamline this for 23 series.

-dhruva

On 3/13/08, Jason Earl <jearl@xmission.com> wrote:
>
> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:
>
> bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
> The other branches are available as well and have URLs like:
>
> http://bzr.notengoamigos.org/emacs/branches/<branch-name>/
>
> For more information please see:
>
> http://bzr.notengoamigos.org/
>
> This repository is obviously something that should mostly interest Emacs
> developers, but I'm sending this email to the bazaar mailing list as
> well in case they are curious.
>
> Jason
>
>
>

-- 
Sent from Gmail for mobile | mobile.google.com

Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Bastien]

dhruva <dhruvakm@gmail.com> writes:

> I request the new maintainers to make a decision in the near future
> and streamline this for 23 series.

AFAIU the decision has been made to use bzr.

The switch doesn't depend on the Emacs maintainers only, it also depends
on savannah maintainers, since the bzr repo has to live there.

Jason's bzr repo is not just "another" repo, it already helps people
testing bzr.  This is useful in any case for all Emacs maintainers.

-- 
Bastien

Re: Emacs Bazaar repository

[Stefan Monnier]

> Well, now we have cvs, git, mercurial and bazaar. Having a subversion

You forgot to mention Arch, which with CVS is the only one actually used
very actively for development (it's used to keep branches in sync).

> repository will complete the long story.
> I just hope this indecision will not continue for too long and confuse
> people in terms of which repository to use. So far, i just had to
> mention 'emacs head' has this problem on some platform in reporting
> bugs. I have now started adding the repo info too!
> I request the new maintainers to make a decision in the near future
> and streamline this for 23 series.

As long as the forgeign branches are kept in sync with the CVS branches,
I don't think it's a big deal.  For me the plan goes as follows:
- ascertain that Bzr is good enough for our usage pattern.
- place Bzr branches that mirror the CVS ones (and are kept in sync)
  on savannah.
- get vc-status to a good enough shape to be usable on those Bzr branches.
- get Miles to switch from Arch to Bzr to do the merge between branches.
- drop the CVS repository.


        Stefan

Re: Emacs Bazaar repository

[dhruva]

Sounds practical. I will start using bazaar baz/bzr(is it bazaar-ng
written in python or the earlier 'c' implementation). For records, I
have tried using all the emacs repositories from cvs,tla,hg,git and
will try bzr/baz too. Command usage and deployment based, tla was
toughest followed by git and cvs. mercurial were easiest to deploy and
use. If it is bzr (python implementation), it will share mercurial's
ease of deployment.

-dhruva

On 3/13/08, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > Well, now we have cvs, git, mercurial and bazaar. Having a subversion
>
> You forgot to mention Arch, which with CVS is the only one actually used
> very actively for development (it's used to keep branches in sync).
>
> > repository will complete the long story.
> > I just hope this indecision will not continue for too long and confuse
> > people in terms of which repository to use. So far, i just had to
> > mention 'emacs head' has this problem on some platform in reporting
> > bugs. I have now started adding the repo info too!
> > I request the new maintainers to make a decision in the near future
> > and streamline this for 23 series.
>
> As long as the forgeign branches are kept in sync with the CVS branches,
> I don't think it's a big deal.  For me the plan goes as follows:
> - ascertain that Bzr is good enough for our usage pattern.
> - place Bzr branches that mirror the CVS ones (and are kept in sync)
>   on savannah.
> - get vc-status to a good enough shape to be usable on those Bzr branches.
> - get Miles to switch from Arch to Bzr to do the merge between branches.
> - drop the CVS repository.
>
>
>         Stefan
>

-- 
Sent from Gmail for mobile | mobile.google.com

Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Karl Fogel]

Jason Earl <jearl@xmission.com> writes:
> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:
>
> bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
> The other branches are available as well and have URLs like:
>
> http://bzr.notengoamigos.org/emacs/branches/<branch-name>/
>
> For more information please see:
>
> http://bzr.notengoamigos.org/

Thanks for doing this.  Two questions:

   1) If we commit to ("push to", whatever the appropriate term is) this
      repository, will the changes show up in CVS?  (I'm assuming CVS is
      still considered the master until some official switchover.)

   2) I run Debian GNU/Linux ("testing" dist) and have installed the
      'bzr' package now using 'apt-get'.  Given the many variants of
      Bazaar(s) that have existed over the years, I can't be positive
      that 'bzr' is the right package; if it is not, someone please post
      here saying what is :-).

-Karl

Re: Emacs Bazaar repository

[Karl Fogel]

Jason Earl <jearl@xmission.com> writes:
> bzr clone http://bzr.notengoamigos.org/emacs/trunk/

By the way, strongly suggest a dest name after that:

   bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr

or something.  Probably no one wants their local clone directory to be
named "trunk" :-).

-Karl

Re: Emacs Bazaar repository

[Jonathan Lange]

On Thu, Mar 13, 2008 at 3:06 PM, Karl Fogel <kfogel@red-bean.com> wrote:
>    2) I run Debian GNU/Linux ("testing" dist) and have installed the
>       'bzr' package now using 'apt-get'.  Given the many variants of
>       Bazaar(s) that have existed over the years, I can't be positive
>       that 'bzr' is the right package; if it is not, someone please post
>       here saying what is :-).
>

'bzr' is the right one. Work is underway to have the package renamed
to 'bazaar'.

jml

Re: Emacs Bazaar repository

[Jonathan Lange]

On Thu, Mar 13, 2008 at 3:09 PM, Karl Fogel <kfogel@red-bean.com> wrote:
> Jason Earl <jearl@xmission.com> writes:
>
> > bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
>  By the way, strongly suggest a dest name after that:
>
>    bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr
>
>  or something.  Probably no one wants their local clone directory to be
>  named "trunk" :-).
>

Actually, it's probably a good idea to do make a shared repository
called 'emacs-bzr' and use the original clone command inside that. So,

  bzr init-repo emacs-bzr
  cd emacs-bzr
  bzr clone http://bzr.notengoamigos.org/emacs/trunk/

That way, Bazaar will share revisions between branches of Emacs,
making it faster to clone other branches.

jml

Re: Emacs Bazaar repository

[Eric Hanchrow]

I tried

  $ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE

bzr ground for about an hour, using 100% of the CPU, then finally failed with

    bzr: ERROR: No such file: 'http://bzr.notengoamigos.org/emacs/.bzr/repository/indices/5b5aedf70f9a908bd9b77d5e921f468f.tix'

-- 
Like most people, I would like to use the words ''parameters''
and ''behoove'' in the same sentence, but I am not sure how.
        -- A Question for 'Ask Mister Language Person'

Re: Emacs Bazaar repository

[Jonathan Lange]

On Thu, Mar 13, 2008 at 3:15 PM, Jonathan Lange <jml@mumak.net> wrote:
>
> On Thu, Mar 13, 2008 at 3:09 PM, Karl Fogel <kfogel@red-bean.com> wrote:
>  > Jason Earl <jearl@xmission.com> writes:
>  >
>  > > bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>  >
>  >  By the way, strongly suggest a dest name after that:
>  >
>  >    bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr
>  >
>  >  or something.  Probably no one wants their local clone directory to be
>  >  named "trunk" :-).
>  >
>
>  Actually, it's probably a good idea to do make a shared repository
>  called 'emacs-bzr' and use the original clone command inside that. So,
>
>   bzr init-repo emacs-bzr
>   cd emacs-bzr
>
>   bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
>  That way, Bazaar will share revisions between branches of Emacs,
>  making it faster to clone other branches.
>

Which reminds me. An even better way to do the initial download is this:

 # Get the tarball
 $ wget http://bzr.notengoamigos.org/emacs.tar.gz
 $ tar xzf emacs.tar.gz

 # Make a repo
 $ bzr init-repo emacs-bzr

 # Seed it with the downloaded branch
 $ cd emacs-bzr
 $ bzr branch ../emacs trunk

(Assuming that the tarball extracts to a directory called 'emacs/').

 # Get the latest changes
 $ cd trunk
 $ bzr pull --remember http://bzr.notengoamigos.org/emacs/trunk/


After you've done this, you'll only need to do 'bzr pull' in the trunk
directory to get the latest changes, and doing 'bzr branch
http://bzr.notengoamigos.org/emacs/branches/<foo>' should be much
faster.

jml

Re: Emacs Bazaar repository

[Jason Earl]

Karl Fogel <kfogel@red-bean.com> writes:

> Jason Earl <jearl@xmission.com> writes:
>> bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
> By the way, strongly suggest a dest name after that:
>
>    bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr
>
> or something.  Probably no one wants their local clone directory to be
> named "trunk" :-).
>
> -Karl

Thanks to some pointers from Jonathan Lange I have updated the
instructions at:

http://bzr.notengoamigos.org/

for using this repository to strongly emphasize getting the tarball of
the shared repository.  If you really want to play with this your first
step should almost certainly be:

wget -c http://bzr.notengoamigos.org/emacs.tar.gz

(or get the lzma compressed version if you'd like).  There are more
instructions on the web site.

There are two reasons for this.  The first is that the process that I
use to update the repository from CVS apparently shifts some of the
packs around.  Unless you have a very fast connection and you time
things just right you are very likely to end up in a situation where
your client bzr is looking for pack files that have moved on my server.
This will cause the client to abort.  That's not a big deal if you are
just trying to pull down the last 20 changesets or so.  Issuing another
"bzr pull" will fix the problem in a couple of seconds.  If you are 95%
of the way through pulling down the 97000 or so changesets in the trunk,
then you'll probably be unhappy.  There probably is a better way to
update the repository so that this doesn't happen, but I am still
figuring some of this stuff out :).

The second reason is that you can't really evaluate the usefulness of
Bazaar without using a shared repository.

Jason

Re: Emacs Bazaar repository

[Jason Earl]

Eric Hanchrow <offby1@blarg.net> writes:

> I tried
>
>   $ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE
>
> bzr ground for about an hour, using 100% of the CPU, then finally failed with
>
>     bzr: ERROR: No such file: 'http://bzr.notengoamigos.org/emacs/.bzr/repository/indices/5b5aedf70f9a908bd9b77d5e921f468f.tix'

The process I use to update the Bazaar repository from CVS apparently
moves some of the needed pack and index files around.  It's possible
that you could go into the directory that was created and do a "bzr
pull" and finish the download, but I am not sure if that always works or
if I just got lucky one time :).

I would suggest that you download the premade repository, cd to the
branches directory and then do try your command again.

$ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE

This will be *much* faster as it will re use the changesets that trunk
and EMACS_22_BASE have in common.  I am going to get in touch with the
bazaar mailing lists and see if there is a better way to do what I am
doing.  It's possible that using the "smart" server would protect you
from this occurrence (I don't know), but the smart server is even slower
when checking out these large branches.  Part of that may be the fact
that my server is not very powerful and using the smart server puts most
of the processing burden on the server end.

I am ccing this to the bazaar mailing list as well to see if anyone
there can help.

Jason

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

I see:

$ bzr --version
Bazaar (bzr) 0.11.0
Using python interpreter: /usr/bin/python
Using python standard library: /usr/lib/python2.4
Using bzrlib: /usr/lib/python2.4/site-packages/bzrlib

$ bzr clone http://bzr.notengoamigos.org/emacs/trunk/ bzr-emacs
bzr: ERROR: Unknown branch format: 'Bazaar Branch Format 6 (bzr 0.15)\n'

What can i do to make this version of bzr to work w/ that repo?

thi

Re: Emacs Bazaar repository

[dhruva]

Hi,
 I got bzr setup and running on my M$-XP box. I got the bzr repo of
emacs (by downloading the emacs.tar.gz) and updated it to the latest.
 I wanted to make sure the recent changesets are pulled in. I typed
"bzr log -l 10". It took ~24 seconds (elapsed time from perfmon)
consistently. Whereas Mercurial (hg log -l 10) on the trunk to 2
seconds (and the mercurial repo is on a CIFS mounted drive while bzr
repo was on local disk). GIT was fastest though even on a CIFS mounted
system.

IMHO, the performance aspects must be considered seriously.

-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Jason Earl]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> I see:
>
> $ bzr --version
> Bazaar (bzr) 0.11.0
> Using python interpreter: /usr/bin/python
> Using python standard library: /usr/lib/python2.4
> Using bzrlib: /usr/lib/python2.4/site-packages/bzrlib
>
> $ bzr clone http://bzr.notengoamigos.org/emacs/trunk/ bzr-emacs
> bzr: ERROR: Unknown branch format: 'Bazaar Branch Format 6 (bzr 0.15)\n'
>
> What can i do to make this version of bzr to work w/ that repo?
>
> thi

Unfortunately you need a newer version of bzr.  I don't think that I
could migrate the current repository to the older format even if I
wanted to do so.

Sorry.

Jason

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

() dhruva <dhruvakm@gmail.com>
() Thu, 13 Mar 2008 13:34:56 +0530

   IMHO, the performance aspects must be considered seriously.

If performance can't be engineered into the entire system, the
usual (serious) way to do it is to use hierarchy to partially
separate the slow parts from the fast parts.

 CPU <-> L[1...n]-CACHE <-> MEMORY <-> DISK <-> NETWORK

Here, CPU can spin as fast as it wants.  It is not limited by
speeds between other elements of the hierarchy, apart from
L1-CACHE, and yet can (if All Goes Well :--) conduct useful
dialogue w/ NETWORK anyway.

I think a similar arrangement is best for my case:

 ttn <-> git <-> bzr <-> emacs@internet

Since i cannot make bzr faster, i do my local spinning w/ Git.
Presently, i'm still (half-heartedly) searching for the git/bzr
gateway, hoping someone will post a url...  btw, i see there is
already some thought on the problem for Git and Mercurial:
<http://thunk.org/tytso/blog/2007/03/24/git-and-hg/>.

For the email-only crowd, on that webpage, Theodore T'so sez:

| So basically git does have short-comings, yes, but people will
| come out in different places about which tools is best for them,
| and that’s OK. Actually, I think the ultimate solution for this
| problem is to build a bidrectoinal hg/git gateway. There are
| tools that will export from hg to git, and vice versa, and they
| are actually pretty sophisticated. I don’t think it should be
| that painful to create a tool that does incremental exports in
| both directions, maintaining state so that the right thing
| happens when a commit gets made on the git side, and gets
| exported into hg, or vice versa. Ultimately I think that’s the
| best solution, since that way people can use whatever tool they
| want, and still contribute and development as first class
| citizens. This is the main reason why I’ve held off on
| converting e2fsprogs to git (although I have made some private
| test repositories which I’ll use to take advantage of git’s
| superior annotation and query/log utilities); I don’t want to
| make a git repository of e2fsprogs public until I’m sure that a
| bidrectional gateway tool won’t require me to make any changes
| that affect the commit-id’s, since that would invalidate any
| work that people have done that was based on a clone of the
| coverted git repository.
|
| I have a rough design for how to do the bidirectional gateway,
| but the issue is finding time to implement it. Anyone with free
| time looking for a project? If so, contact me. I probably should
| have written this up as a potential Google Summer of Code
| project, but it’s too late for this year. Oh, well.

IMHO, it's just a matter of time until some Righteous Hacker
stitches it all together.  It's a nice problem.

thi

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

() Jason Earl <jearl@xmission.com>
() Thu, 13 Mar 2008 02:49:06 -0600

   Unfortunately you need a newer version of bzr.  I don't think that I
   could migrate the current repository to the older format even if I
   wanted to do so.

OK, will try a newer bzr (grumble grumble).

thi

Re: Emacs Bazaar repository

[David Kastrup]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () dhruva <dhruvakm@gmail.com>
> () Thu, 13 Mar 2008 13:34:56 +0530
>
>    IMHO, the performance aspects must be considered seriously.
>
> If performance can't be engineered into the entire system, the
> usual (serious) way to do it is to use hierarchy to partially
> separate the slow parts from the fast parts.
>
>  CPU <-> L[1...n]-CACHE <-> MEMORY <-> DISK <-> NETWORK
>
> Here, CPU can spin as fast as it wants.  It is not limited by
> speeds between other elements of the hierarchy, apart from
> L1-CACHE, and yet can (if All Goes Well :--) conduct useful
> dialogue w/ NETWORK anyway.
>
> I think a similar arrangement is best for my case:
>
>  ttn <-> git <-> bzr <-> emacs@internet
>
> Since i cannot make bzr faster, i do my local spinning w/ Git.
> Presently, i'm still (half-heartedly) searching for the git/bzr
> gateway, hoping someone will post a url...

There is something called tailor or taylor.

-- 
David Kastrup

Re: Emacs Bazaar repository

[Andreas Schwab]

"Jonathan Lange" <jml@mumak.net> writes:

> Actually, it's probably a good idea to do make a shared repository
> called 'emacs-bzr' and use the original clone command inside that. So,
>
>   bzr init-repo emacs-bzr
>   cd emacs-bzr
>   bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
> That way, Bazaar will share revisions between branches of Emacs,
> making it faster to clone other branches.

Can you do that sharing also between different directories, that is the
equivalent of git clone --reference?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Juanma Barranquero]

On Thu, Mar 13, 2008 at 5:09 AM, Karl Fogel <kfogel@red-bean.com> wrote:

> Probably no one wants their local clone directory to be
>  named "trunk" :-).

My CVS checkout of Emacs' trunk is in a directory named "trunk"... :)

(But I agree it is a good idea to suggest a directory name)

 Juanma

Re: Emacs Bazaar repository

[James Westby]

On Wed, 2008-03-12 at 23:20 -0600, Jason Earl wrote:
> Eric Hanchrow <offby1@blarg.net> writes:
> 
> > I tried
> >
> >   $ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE
> >
> > bzr ground for about an hour, using 100% of the CPU, then finally failed with
> >
> >     bzr: ERROR: No such file: 'http://bzr.notengoamigos.org/emacs/.bzr/repository/indices/5b5aedf70f9a908bd9b77d5e921f468f.tix'
> 
> The process I use to update the Bazaar repository from CVS apparently
> moves some of the needed pack and index files around.  It's possible
> that you could go into the directory that was created and do a "bzr
> pull" and finish the download, but I am not sure if that always works or
> if I just got lucky one time :).

Hi,

Can you tell us what that process is?

(Apologies if I missed that in previous mails).

> I would suggest that you download the premade repository, cd to the
> branches directory and then do try your command again.
> 
> $ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE
> 
> This will be *much* faster as it will re use the changesets that trunk
> and EMACS_22_BASE have in common.  I am going to get in touch with the
> bazaar mailing lists and see if there is a better way to do what I am
> doing.  It's possible that using the "smart" server would protect you
> from this occurrence (I don't know), but the smart server is even slower
> when checking out these large branches.  Part of that may be the fact
> that my server is not very powerful and using the smart server puts most
> of the processing burden on the server end.

If this is caused by what I think it is then the smart server won't
help unfortunately.

I can recommend that everyone sets up a shared repository for 
themselves (bzr init-repo dir), it will massively reduce disk usage
and time for some operations.

You need to do this before creating a branch, and then create the branch
inside the directory you create (the dir argument to init-repo). Then
all branches that you create under there will share storage where 
possible.

Jason, did you create these branches in a rich-root-pack format 
repository, or just pack-0.92 (the default with 1.0 or later)?

Thanks,

James

Re: Emacs Bazaar repository

[Andreas Schwab]

My first impression is that bzr is slow, so slow that it is completely
unusable.  How can it come that a simple bzr log takes more than a
minute to even start?  Even cvs log is instantaneous in comparison,
although it has to request the log from the server.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[David Kastrup]

Andreas Schwab <schwab@suse.de> writes:

> My first impression is that bzr is slow, so slow that it is completely
> unusable.  How can it come that a simple bzr log takes more than a
> minute to even start?  Even cvs log is instantaneous in comparison,
> although it has to request the log from the server.

I find this surprising: "git log" is pretty much instantaneous, and git
recalculates a code piece's history in the process (renaming and copying
is not tracked by git, but calculated on the fly when you ask for log
output).  In contrast, one has to tell Bazaar IIRC when one copies or
moves or renames files, so it should have the information available
right away.

-- 
David Kastrup

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

() David Kastrup <dak@gnu.org>
() Thu, 13 Mar 2008 10:41:45 +0100

   There is something called tailor or taylor.

Thanks for the tip.  I have found its homepage:
http://progetti.arstecnica.it/tailor/wiki

thi

Re: Emacs Bazaar repository

[Andrea Russo]

Jason Earl <jearl@xmission.com> writes:

> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.

Thank you.

> If anyone is interested in playing with the results you can check out
> the HEAD of the CVS repository with:
>
> bzr clone http://bzr.notengoamigos.org/emacs/trunk/

I tried with:

bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr

I ran it for 8 hours over a 2Mbit ADSL line then got bored (because it
was crunching my hard disk) a killed it. The bzr status bar was around
70%.

I used to keep in sync my Emacs repository from Miles Arch repo and IIRC
the inital copy took much less time.

Recently I switched to git using `git://git.sv.gnu.org/emacs.git' and
the clone operation took less than an hour.

Regards,
Andrea.

--
Do not worry about your difficulties in mathematics;
I can assure you that mine are still greater.
                                  -- Albert Einstein

Re: Emacs Bazaar repository

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jason Earl wrote:
> Unfortunately you need a newer version of bzr.  I don't think that I
> could migrate the current repository to the older format even if I
> wanted to do so.

I don't think it would be particularly hard to convert it to the "knit"
format that was used in 0.11, but I think it would be a pretty bad idea.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFH2SgG0F+nu1YWqI0RAk4AAJdg/i7dMIDqyN8JM3Fjw3EPBOKTAJ0aNXzd
wWFsEI7+CXrMh1Es2gcoCQ==
=bRvU
-----END PGP SIGNATURE-----

Re: Emacs Bazaar repository

[Andreas Schwab]

Jason Earl <jearl@xmission.com> writes:

> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:
>
> bzr clone http://bzr.notengoamigos.org/emacs/trunk/

It looks like all commit dates are off by 4 hours.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Stefan Monnier]

> My CVS checkout of Emacs' trunk is in a directory named "trunk"... :)

Same here,


        Stefan

Re: Emacs Bazaar repository

[Karl Fogel]

Never got an answer to this question:

Karl Fogel <kfogel@red-bean.com> writes:
>    1) If we commit to ("push to", whatever the appropriate term is) this
>       repository, will the changes show up in CVS?  (I'm assuming CVS is
>       still considered the master until some official switchover.)

It'd be good to know where to commit... :-)

Thanks,
-Karl

Re: Emacs Bazaar repository

[dhruva]

Hi,
 From what little I know, there is no bidirectional sync support at
the moment. CVS still appears to be the official repository into which
someone with write access should push the changes. This is based on my
observations of miles getting his code from Arch to CVS.

-dhruva

On Thu, Mar 13, 2008 at 9:00 PM, Karl Fogel <kfogel@red-bean.com> wrote:
> Never got an answer to this question:
>
>
>  Karl Fogel <kfogel@red-bean.com> writes:
>  >    1) If we commit to ("push to", whatever the appropriate term is) this
>  >       repository, will the changes show up in CVS?  (I'm assuming CVS is
>  >       still considered the master until some official switchover.)
>
>  It'd be good to know where to commit... :-)
>
>  Thanks,
>  -Karl
>
>
>



-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Jason Earl]

James Westby <jw+debian@jameswestby.net> writes:

> On Wed, 2008-03-12 at 23:20 -0600, Jason Earl wrote:
>> Eric Hanchrow <offby1@blarg.net> writes:
>> 
>> > I tried
>> >
>> >   $ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE
>> >
>> > bzr ground for about an hour, using 100% of the CPU, then finally failed with
>> >
>> >     bzr: ERROR: No such file: 'http://bzr.notengoamigos.org/emacs/.bzr/repository/indices/5b5aedf70f9a908bd9b77d5e921f468f.tix'
>> 
>> The process I use to update the Bazaar repository from CVS apparently
>> moves some of the needed pack and index files around.  It's possible
>> that you could go into the directory that was created and do a "bzr
>> pull" and finish the download, but I am not sure if that always works or
>> if I just got lucky one time :).
>
> Hi,
>
> Can you tell us what that process is?
>
> (Apologies if I missed that in previous mails).
>
>> I would suggest that you download the premade repository, cd to the
>> branches directory and then do try your command again.
>> 
>> $ bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE
>> 
>> This will be *much* faster as it will re use the changesets that trunk
>> and EMACS_22_BASE have in common.  I am going to get in touch with the
>> bazaar mailing lists and see if there is a better way to do what I am
>> doing.  It's possible that using the "smart" server would protect you
>> from this occurrence (I don't know), but the smart server is even slower
>> when checking out these large branches.  Part of that may be the fact
>> that my server is not very powerful and using the smart server puts most
>> of the processing burden on the server end.
>
> If this is caused by what I think it is then the smart server won't
> help unfortunately.
>
> I can recommend that everyone sets up a shared repository for
> themselves (bzr init-repo dir), it will massively reduce disk usage
> and time for some operations.

My original instructions didn't stress using a shared repository, but my
current ones do.  In fact, my current instructions suggest downloading a
premade shared repository.

> You need to do this before creating a branch, and then create the
> branch inside the directory you create (the dir argument to
> init-repo). Then all branches that you create under there will share
> storage where possible.
>
> Jason, did you create these branches in a rich-root-pack format 
> repository, or just pack-0.92 (the default with 1.0 or later)?

I used the default pack-0.92 format.  I honestly wasn't sure which
format was the most appropriate.

Jason

Re: Emacs Bazaar repository

[Jason Earl]

Karl Fogel <kfogel@red-bean.com> writes:

> Never got an answer to this question:
>
> Karl Fogel <kfogel@red-bean.com> writes:
>>    1) If we commit to ("push to", whatever the appropriate term is) this
>>       repository, will the changes show up in CVS?  (I'm assuming CVS is
>>       still considered the master until some official switchover.)
>
> It'd be good to know where to commit... :-)
>
> Thanks,
> -Karl

I'm just some random guy that lurks on emacs-devel :).  I set up a test
repository because A) I like playing with version control systems, B) I
thought it would be helpful, and C) I was curious to see if it could
even be done.

In short, this repository is about as official as a $3 bill.  Someone
mentioned that a Bazaar repository should be set up, and I just happened
to be experimenting with Bazaar using the Emacs code base.

Jason

How to spell "commit" [was: bikeshedding bzr (or similar) :]

[Stephen J. Turnbull]

[[ As long as I'm here dept: Jason, you have my admiration, thanks,
and congratulations for following up on that:  I have not yet had the
patience to follow through on a cvsps conversion, though I've tried
several times! ]]

Wow, among all the bikeshedding[1][2], a real question that matters.

Karl Fogel writes:

 > Karl Fogel <kfogel@red-bean.com> writes:
 > >    1) If we commit to ("push to", whatever the appropriate term is)

Darcs's command set is the best terminology to use here, I think.

You *record* a changeset locally.

You *push* a changeset to make it public.

People are going to call both "commits", of course, but the record
vs. push terminology is nonetheless unambiguous and reasonably
intuitive.  I think it will grow on people if we just start using it.


Footnotes: 
[1]  Thien-Thi's question about maintaining personal gateways among
VCSes is also an important question.  I use tailor myself for over a
year now; it has some problems, but it overall works well.

[2]  Of course efficiency will matter, but *not yet*, for heaven's
sake!  "Premature optimization is the root of all evil in programming"!

Re: Emacs Bazaar repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Date: Thu, 13 Mar 2008 12:43:42 +0100
> Cc: bazaar@lists.canonical.com, emacs-devel@gnu.org
> 
> Even cvs log is instantaneous in comparison,

How do you mean ``instantaneous''?  For me, it takes 10 seconds to
start displaying the log.

Re: Emacs Bazaar repository

[Jonathan Lange]

On Thu, Mar 13, 2008 at 8:58 PM, Andreas Schwab <schwab@suse.de> wrote:
> "Jonathan Lange" <jml@mumak.net> writes:
>
>
> > Actually, it's probably a good idea to do make a shared repository
>  > called 'emacs-bzr' and use the original clone command inside that. So,
>  >
>  >   bzr init-repo emacs-bzr
>  >   cd emacs-bzr
>  >   bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>  >
>  > That way, Bazaar will share revisions between branches of Emacs,
>  > making it faster to clone other branches.
>
>  Can you do that sharing also between different directories, that is the
>  equivalent of git clone --reference?
>

I'm not exactly sure what you mean, since I don't know much about git.
I'll tell you what I know, instead.

In that example, any branches beneath the 'emacs-bzr' will share the
repository in 'emacs-bzr'. In Bazaar, branches almost always have
their own directory.

jml

Re: Emacs Bazaar repository

[Jonathan Lange]

On Thu, Mar 13, 2008 at 7:04 PM, dhruva <dhruvakm@gmail.com> wrote:
> Hi,
>   I got bzr setup and running on my M$-XP box. I got the bzr repo of
>  emacs (by downloading the emacs.tar.gz) and updated it to the latest.
>   I wanted to make sure the recent changesets are pulled in. I typed
>  "bzr log -l 10". It took ~24 seconds (elapsed time from perfmon)
>  consistently. Whereas Mercurial (hg log -l 10) on the trunk to 2
>  seconds (and the mercurial repo is on a CIFS mounted drive while bzr
>  repo was on local disk). GIT was fastest though even on a CIFS mounted
>  system.
>

I was a bit curious about this so I asked my friendly neighbourhood
Bazaar hacker (CCd). He tells me that comparing 'bzr log' to 'hg log'
is like comparing apples with ... umm, apple trees. Bazaar's log
displays all merges and thus has to do calculations on the whole
ancestry graph. Mercurial's log just displays the mainline history.
'bzr log --line' is a better comparison.

Unfortunately, 'bzr log --line' is nowhere near as fast as it could
be. We did a quick test and saw that 'log --line -l 10' takes about
half the time as 'log -l 10' which is still too slow. It's still doing
too much work with the whole graph.

We whipped up a quick hack and got it doing the same thing in about
0.25 seconds, which is getting close.

>  IMHO, the performance aspects must be considered seriously.
>

Definitely. I hate waiting for anything, computers most of all. I'll
hassle Martin and try to get this fixed. Alternatively, the Bazaar
project welcomes patches with open arms. :)

jml

Re: Emacs Bazaar repository

[Dan Nicolaescu]

"Jonathan Lange" <jml@mumak.net> writes:

  > On Thu, Mar 13, 2008 at 3:15 PM, Jonathan Lange <jml@mumak.net> wrote:
  > >
  > > On Thu, Mar 13, 2008 at 3:09 PM, Karl Fogel <kfogel@red-bean.com> wrote:
  > >  > Jason Earl <jearl@xmission.com> writes:
  > >  >
  > >  > > bzr clone http://bzr.notengoamigos.org/emacs/trunk/
  > >  >
  > >  >  By the way, strongly suggest a dest name after that:
  > >  >
  > >  >    bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr
  > >  >
  > >  >  or something.  Probably no one wants their local clone directory to be
  > >  >  named "trunk" :-).
  > >  >
  > >
  > >  Actually, it's probably a good idea to do make a shared repository
  > >  called 'emacs-bzr' and use the original clone command inside that. So,
  > >
  > >   bzr init-repo emacs-bzr
  > >   cd emacs-bzr
  > >
  > >   bzr clone http://bzr.notengoamigos.org/emacs/trunk/
  > >
  > >  That way, Bazaar will share revisions between branches of Emacs,
  > >  making it faster to clone other branches.
  > >
  > 
  > Which reminds me. An even better way to do the initial download is this:
  > 
  >  # Get the tarball
  >  $ wget http://bzr.notengoamigos.org/emacs.tar.gz
  >  $ tar xzf emacs.tar.gz
  > 
  >  # Make a repo
  >  $ bzr init-repo emacs-bzr
  > 
  >  # Seed it with the downloaded branch
  >  $ cd emacs-bzr
  >  $ bzr branch ../emacs trunk
                  ^^^^^^^^^
Is this actually  ../emacs/trunk  ?

(Could you please update the website with these instructions? A good
place would be after the place where you talk about downloading the
.tar.gz file, they are very useful)

BTW, how fast is "bzr annotate" supposed to be?

time bzr annotate vc.el >& /dev/null
133.673u 2.392s 2:23.78 94.6%   0+0k 39864+16io 1pf+0w

The similar command for  Emacs CVS HEAD:
time cvs annotate vc.el >& /dev/null
0.250u 0.064s 0:03.22 9.6%      0+0k 0+0io 0pf+0w

Thanks a lot for doing all this!

Re: Emacs Bazaar repository

[Jason Earl]

Dan Nicolaescu <dann@ics.uci.edu> writes:

> "Jonathan Lange" <jml@mumak.net> writes:
>
>   > On Thu, Mar 13, 2008 at 3:15 PM, Jonathan Lange <jml@mumak.net> wrote:
>   > >
>   > > On Thu, Mar 13, 2008 at 3:09 PM, Karl Fogel <kfogel@red-bean.com> wrote:
>   > >  > Jason Earl <jearl@xmission.com> writes:
>   > >  >
>   > >  > > bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>   > >  >
>   > >  >  By the way, strongly suggest a dest name after that:
>   > >  >
>   > >  >    bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr
>   > >  >
>   > >  >  or something.  Probably no one wants their local clone directory to be
>   > >  >  named "trunk" :-).
>   > >  >
>   > >
>   > >  Actually, it's probably a good idea to do make a shared repository
>   > >  called 'emacs-bzr' and use the original clone command inside that. So,
>   > >
>   > >   bzr init-repo emacs-bzr
>   > >   cd emacs-bzr
>   > >
>   > >   bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>   > >
>   > >  That way, Bazaar will share revisions between branches of Emacs,
>   > >  making it faster to clone other branches.
>   > >
>   > 
>   > Which reminds me. An even better way to do the initial download is this:
>   > 
>   >  # Get the tarball
>   >  $ wget http://bzr.notengoamigos.org/emacs.tar.gz
>   >  $ tar xzf emacs.tar.gz
>   > 
>   >  # Make a repo
>   >  $ bzr init-repo emacs-bzr
>   > 
>   >  # Seed it with the downloaded branch
>   >  $ cd emacs-bzr
>   >  $ bzr branch ../emacs trunk
>                   ^^^^^^^^^
> Is this actually  ../emacs/trunk  ?

No, Jonathan didn't realize that the tarball already contains a shared
repository.  There is no need to create a repository for yourself if you
download the tarball.

> (Could you please update the website with these instructions? A good
> place would be after the place where you talk about downloading the
> .tar.gz file, they are very useful)

The website is as it stands is correct.

> BTW, how fast is "bzr annotate" supposed to be?
>
> time bzr annotate vc.el >& /dev/null
> 133.673u 2.392s 2:23.78 94.6%   0+0k 39864+16io 1pf+0w
>
> The similar command for  Emacs CVS HEAD:
> time cvs annotate vc.el >& /dev/null
> 0.250u 0.064s 0:03.22 9.6%      0+0k 0+0io 0pf+0w
>
> Thanks a lot for doing all this!

That's about how long it took on my machine as well.

Jason

Re: Emacs Bazaar repository

[dhruva]

Hi,


On Fri, Mar 14, 2008 at 5:16 AM, Jonathan Lange <jml@mumak.net> wrote:
>  Definitely. I hate waiting for anything, computers most of all. I'll
>  hassle Martin and try to get this fixed. Alternatively, the Bazaar
>  project welcomes patches with open arms. :)

For starters, could anyone please suggest if there is some magic flag
to profile the bzr code? If I can get the time spent in each function
(PYTHON), we would be able to track the real hot spots. I am open to
help in debugging with my current limited knowledge of bzr internals.

-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Jason Earl]

dhruva <dhruvakm@gmail.com> writes:

> Hi,
>
>
> On Fri, Mar 14, 2008 at 5:16 AM, Jonathan Lange <jml@mumak.net> wrote:
>>  Definitely. I hate waiting for anything, computers most of all. I'll
>>  hassle Martin and try to get this fixed. Alternatively, the Bazaar
>>  project welcomes patches with open arms. :)
>
> For starters, could anyone please suggest if there is some magic flag
> to profile the bzr code? If I can get the time spent in each function
> (PYTHON), we would be able to track the real hot spots. I am open to
> help in debugging with my current limited knowledge of bzr internals.
>
> -dhruva

I can profile by bzr with hotshot using:

$ bzr log --profile

I haven't tried it but apparently:

$ bzr log --lsprof

does something as well.

Jason

Re: Emacs Bazaar repository

[Martin Pool]

On 14/03/2008, dhruva <dhruvakm@gmail.com> wrote:
>  On Fri, Mar 14, 2008 at 5:16 AM, Jonathan Lange <jml@mumak.net> wrote:
>  >  Definitely. I hate waiting for anything, computers most of all. I'll
>  >  hassle Martin and try to get this fixed. Alternatively, the Bazaar
>  >  project welcomes patches with open arms. :)
>
> For starters, could anyone please suggest if there is some magic flag
>  to profile the bzr code? If I can get the time spent in each function
>  (PYTHON), we would be able to track the real hot spots. I am open to
>  help in debugging with my current limited knowledge of bzr internals.

That would be great, see

http://doc.bazaar-vcs.org/bzr.dev/developers/profiling.html
and
http://doc.bazaar-vcs.org/bzr.dev/en/developer-guide/HACKING.html

I believe the main problem is that we are processing the whole graph
to work out which revisions merged which others.  (which hg and git do
not do, iiuc.)  For even quite active trees this is good, but emacs
has a lot of history.

In this case you probably just want the last ten direct commits, bzr
log --line -l10.  mwh posted a patch to improve this and there is some
more that can be done.

-- 
Martin

Re: Emacs Bazaar repository

[Forest Bond]

[-- Attachment #1: Type: text/plain, Size: 551 bytes --]

Hi,

On Thu, Mar 13, 2008 at 12:43:42PM +0100, Andreas Schwab wrote:
> My first impression is that bzr is slow, so slow that it is completely
> unusable.  How can it come that a simple bzr log takes more than a
> minute to even start?  Even cvs log is instantaneous in comparison,
> although it has to request the log from the server.

You'll want to make sure you are using at least version 0.92; preferably, use
something 1.0 or later.  Performance prior to 0.92 was not great.

-Forest
-- 
Forest Bond
http://www.alittletooquiet.net

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Jonathan Lange]

On Thu, Mar 13, 2008 at 10:43 PM, Andreas Schwab <schwab@suse.de> wrote:
> My first impression is that bzr is slow, so slow that it is completely
>  unusable.  How can it come that a simple bzr log takes more than a
>  minute to even start?  Even cvs log is instantaneous in comparison,
>  although it has to request the log from the server.
>

As I mentioned in an earlier post, 'bzr log' and 'cvs log' are doing
very different things. 'bzr log --line' is much closer to typical 'cvs
log' behaviour.

Also, there's now a patch for Bazaar to make 'bzr log --line' a lot
faster -- http://bundlebuggy.aaronbentley.com/request/%3C47D9E60A.6030203%40canonical.com%3E

jml

Re: Emacs Bazaar repository

[Jonathan Lange]

On Fri, Mar 14, 2008 at 12:51 PM, Jason Earl <jearl@xmission.com> wrote:
>
> Dan Nicolaescu <dann@ics.uci.edu> writes:
>  > BTW, how fast is "bzr annotate" supposed to be?
>  >
>  > time bzr annotate vc.el >& /dev/null
>  > 133.673u 2.392s 2:23.78 94.6%   0+0k 39864+16io 1pf+0w
>  >
>  > The similar command for  Emacs CVS HEAD:
>  > time cvs annotate vc.el >& /dev/null
>  > 0.250u 0.064s 0:03.22 9.6%      0+0k 0+0io 0pf+0w
>  >
>  > Thanks a lot for doing all this!
>
>  That's about how long it took on my machine as well.
>

Bazaar recently made annotate slower while making a lot of other
operations much faster. I haven't been following it closely, but I
know that John Arbash Meinel has been working hard on bringing
annotate up to speed. Expect significant improvements soon.

jml

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

Martin Pool writes:

 > I believe the main problem is that we are processing the whole graph
 > to work out which revisions merged which others.

What does this mean?  That you can avoid pulling patches that cause
conflicts if they've already been applied?  But that's not terribly
interesting in most logging or mass download applications.

While I quoted Hoare on the root of all evil elsewhere, and continue
to take that as my default position, these numbers are uncomfortable.
I don't care how cold the CPU cache is, I don't want an attempt to
access a non-existent local repo to take 10s.  Nor should a
disk-to-disk copy of a 400MB directory tree take 23.5m.  The partial
clone times are heartening, although it's hard to guess what they
mean in terms of throughput.

# Initial one-branch clone.
$ time bzr clone http://bzr.notengoamigos.org/emacs/trunk/ earl-bzr
Branched 87515 revision(s).                                                     

real    101m35.490s	user    28m16.698s	sys     2m7.827s

# Clone from a local standalone branch into a shared repo.
$ time bzr clone ../earl-bzr trunk
Branched 87515 revision(s).                                                     

real    23m27.626s	user    13m57.869s	sys     1m12.516s

# Command line typo on a local URL, bzr figures it out in 10s.
$ time bzr branch EMACS_22_BASE
bzr: ERROR: Not a branch: "/Users/steve/Software/Emacs/emacs-bzr/EMACS_22_BASE/".

real    0m10.183s	user    0m0.457s	sys     0m1.361s

# Command line typo on a remote URL, bzr figures it out in 3.5s.
$ time bzr clone http://bzr.notengoamigos.org/emacs/EMACS_22_BASE
bzr: ERROR: Not a branch: "http://bzr.notengoamigos.org/emacs/EMACS_22_BASE/".

real    0m3.427s	user    0m0.499s	sys     0m1.142s

# Command line typo on a remote URL, bzr figures it out in 2s.
$ time bzr clone http://bzr.notengoamigos.org/emacs/branch/EMACS_22_BASE
bzr: ERROR: Not a branch: "http://bzr.notengoamigos.org/emacs/branch/EMACS_22_BASE/".

real    0m1.777s	user    0m0.480s	sys     0m0.995s

# A partial clone into the shared repo.
$ time bzr clone http://bzr.notengoamigos.org/emacs/branches/EMACS_22_BASE
Branched 83329 revision(s).                                                     

real    4m47.853s	user    2m20.361s	sys     0m25.168s

# A second partial clone into the shared repo.
$ time bzr clone http://bzr.notengoamigos.org/emacs/branches/lexbind      
Branched 48705 revision(s).                                                     

real    11m18.166s	user    3m25.059s	sys     0m33.614s

Re: Emacs Bazaar repository

[Dan Nicolaescu]

"Jonathan Lange" <jml@mumak.net> writes:

  > On Thu, Mar 13, 2008 at 10:43 PM, Andreas Schwab <schwab@suse.de> wrote:
  > > My first impression is that bzr is slow, so slow that it is completely
  > >  unusable.  How can it come that a simple bzr log takes more than a
  > >  minute to even start?  Even cvs log is instantaneous in comparison,
  > >  although it has to request the log from the server.
  > >
  > 
  > As I mentioned in an earlier post, 'bzr log' and 'cvs log' are doing
  > very different things. 'bzr log --line' is much closer to typical 'cvs
  > log' behaviour.

It is not that close, "cvs log" shows the whole change long for each
entry, not just a single line. For emacs we have very well written
change logs, and if you look at them, you actually want to see the
details. 
Is there a fast alternative that shows the whole entry, not just a
single line?

Also, is bzr status expected to be fast? 

bzr status FILENAME

takes 1.2 seconds on a slow machine and .4 seconds on a fast one. Both
numbers seem quite high.

Re: Emacs Bazaar repository

[Alexander Belchenko]

Dan Nicolaescu пишет:
> "Jonathan Lange" <jml@mumak.net> writes:
> 
>   > On Thu, Mar 13, 2008 at 10:43 PM, Andreas Schwab <schwab@suse.de> wrote:
>   > > My first impression is that bzr is slow, so slow that it is completely
>   > >  unusable.  How can it come that a simple bzr log takes more than a
>   > >  minute to even start?  Even cvs log is instantaneous in comparison,
>   > >  although it has to request the log from the server.
>   > >
>   > 
>   > As I mentioned in an earlier post, 'bzr log' and 'cvs log' are doing
>   > very different things. 'bzr log --line' is much closer to typical 'cvs
>   > log' behaviour.
> 
> It is not that close, "cvs log" shows the whole change long for each
> entry, not just a single line. For emacs we have very well written
> change logs, and if you look at them, you actually want to see the
> details. 
> Is there a fast alternative that shows the whole entry, not just a
> single line?

Try: bzr log --short

> 
> Also, is bzr status expected to be fast? 
> 
> bzr status FILENAME
> 
> takes 1.2 seconds on a slow machine and .4 seconds on a fast one. Both
> numbers seem quite high.
> 
> 
> 

Re: Emacs Bazaar repository

[Martin Pool]

On 14/03/2008, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Martin Pool writes:
>
>   > I believe the main problem is that we are processing the whole graph
>   > to work out which revisions merged which others.
>
> What does this mean?  That you can avoid pulling patches that cause
>  conflicts if they've already been applied?  But that's not terribly
>  interesting in most logging or mass download applications.

Bazaar shows revisions in a nested form like this:

  a
  b
    c
    d
  e
  f
  ..

This means that c and d were merged into this branch in revision b,
and were not previously present in the repository.  This layout gives
a good overview of the history, in our opinion a much better one than
what you get from tools that just print one path through the graph or
that print the revisions in arbitrary linear order.

To calculate which revisions are newly merged is currently done by
examining the whole graph.  We will do two things to speed it up:
faster loading of the graph from the repository, and caching some data
in the branch.

But possibly you don't want to know about merged in revisions, and
that can be done with bzr log --short or --line.  These should be able
to do much less work, just walking down through the revision to print.
mwh posted a patch for this, and we can take it further.

>  While I quoted Hoare on the root of all evil elsewhere, and continue
>  to take that as my default position, these numbers are uncomfortable.
>  I don't care how cold the CPU cache is, I don't want an attempt to
>  access a non-existent local repo to take 10s.  Nor should a
>  disk-to-disk copy of a 400MB directory tree take 23.5m.  The partial
>  clone times are heartening, although it's hard to guess what they
>  mean in terms of throughput.

I want to point out that bzr branch locally, not in a single
repository, is doing more than just copying all the files - it is
filtering out and verifying the data relevant to the particular branch
you're copying.  If you just want to copy the data, use cp; if you
want a new branch without copying, use init-repo to create a common
repository directory above them.

That said, it would probably be useful to have an option which copies
the whole thing with no or fewer checks, and to speed up the way we do
copy it.

>  # Initial one-branch clone.
>  $ time bzr clone http://bzr.notengoamigos.org/emacs/trunk/ earl-bzr
>  Branched 87515 revision(s).
>
>  real    101m35.490s     user    28m16.698s      sys     2m7.827s
>
>  # Clone from a local standalone branch into a shared repo.
>  $ time bzr clone ../earl-bzr trunk
>  Branched 87515 revision(s).
>
>  real    23m27.626s      user    13m57.869s      sys     1m12.516s
>
>  # Command line typo on a local URL, bzr figures it out in 10s.
>  $ time bzr branch EMACS_22_BASE
>  bzr: ERROR: Not a branch: "/Users/steve/Software/Emacs/emacs-bzr/EMACS_22_BASE/".
>
>  real    0m10.183s       user    0m0.457s        sys     0m1.361s

Maybe you could send (perhaps just on the bzr list) a run of that with
"bzr --lsprof branch ...".  It may be that the low cpu usage is caused
by many things being pushed out of ram by building the emacs working
tree in the previous command.

>  # Command line typo on a remote URL, bzr figures it out in 3.5s.
>  $ time bzr clone http://bzr.notengoamigos.org/emacs/EMACS_22_BASE
>  bzr: ERROR: Not a branch: "http://bzr.notengoamigos.org/emacs/EMACS_22_BASE/".
>
>  real    0m3.427s        user    0m0.499s        sys     0m1.142s

For me this remote urls takes about .6s; I may be closer to that
server.  At any rate it is only sending one http request which is the
least that can be reasonably expected. :)

-- 
Martin

Re: Emacs Bazaar repository

[Dan Nicolaescu]

Alexander Belchenko <bialix@ukr.net> writes:

  > Dan Nicolaescu пишет:
  > > "Jonathan Lange" <jml@mumak.net> writes:
  > >
  > >   > On Thu, Mar 13, 2008 at 10:43 PM, Andreas Schwab <schwab@suse.de> wrote:
  > >   > > My first impression is that bzr is slow, so slow that it is completely
  > >   > >  unusable.  How can it come that a simple bzr log takes more than a
  > >   > >  minute to even start?  Even cvs log is instantaneous in comparison,
  > >   > >  although it has to request the log from the server.
  > >   > >
  > >   >   > As I mentioned in an earlier post, 'bzr log' and 'cvs log'
  > > are doing
  > >   > very different things. 'bzr log --line' is much closer to typical 'cvs
  > >   > log' behaviour.
  > >
  > > It is not that close, "cvs log" shows the whole change long for each
  > > entry, not just a single line. For emacs we have very well written
  > > change logs, and if you look at them, you actually want to see the
  > > details. Is there a fast alternative that shows the whole entry, not
  > > just a
  > > single line?
  > 
  > Try: bzr log --short

The content is OK, but it takes 30 seconds to run 
bzr log --limit=30 --short FILENAME

Re: Emacs Bazaar repository

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eli Zaretskii wrote:
>> From: Matthieu Moy <Matthieu.Moy@imag.fr>
>> Date: Fri, 14 Mar 2008 10:58:13 +0100
>> Cc: Andreas Schwab <schwab@suse.de>, emacs-devel@gnu.org,
>> 	bazaar@lists.canonical.com
>>
>>> Andreas Schwab <schwab@suse.de> writes:
>>>
>>>> My first impression is that bzr is slow, so slow that it is completely
>>>> unusable.  How can it come that a simple bzr log takes more than a
>>>> minute to even start?  Even cvs log is instantaneous in comparison,
>>>> although it has to request the log from the server.
>> [...]
>> As opposed to that, bzr has to get a global view of history at least
>> to get the revision numbers (there was some plans caching this
>> information, I don't know what's the status).
>>
>> That said, the time for bzr log to start should clearly not be _that_
>> long.
> 
> Incidentally, why are we concentrating on "bzr log"? is that a
> frequent operation?  With CVS, I find myself doing "cvs log" only once
> in a few months, when I'm looking for a change corresponding to some
> ChangeLog entry.
> 
> Aren't "push" and "pull" much more important, as far as speed is
> concerned, for everyday work?  Also the equivalent of "cvs diff", I
> think.  Those are the ops I use much more frequently than "log" and
> "annotate".

I think it is because 'bzr log' is surprisingly slow when compared to
other systems, while the other commands are not.

The biggest reason 'bzr log' is slow is because we spend some time
analyzing the ancestry to give a "pretty" view, while git/hg do not.
Specifically, when you do "bzr log" we traverse the ancestry to figure
out when revisions were merged, etc.

I believe plain "git log" just starts outputting the revisions as it
encounters them, and "hg log" also outputs them as they are stored. In
the case of "hg" that means that the order of operations you did to
create the branch will change the order you see them displayed. So if
you and someone else do 2 commits and then you merge them, and they pull
you. At that point "hg log" gives different results, even though both
branches are "the same".

For example:

hg init A
cd A
echo foo > foo
hg add
hg commit -m 'init'
cd ..
hg clone A B
cd A
echo "A1" >> foo
hg commit -m "A1"
echo "A2" >> foo
hg commit -m "A2"
cd ../B
echo "B1" > bar
hg add
hg commit -m "B1"
echo "B2" >> bar
hg commit -m "B2"
hg pull ../A
hg merge
hg commit -m "Merged"
cd ../A
hg pull -u ../B

At that point doing:
cd A
hg log

gives me: (paraphrased for clarity)
changeset:   5:04157da570b3
summary:     Merged

changeset:   4:e44ea38fbbb9
summary:     B2

changeset:   3:d2df37f438ac
summary:     B1

changeset:   2:111f2448422b
summary:     A2

changeset:   1:3a8ecee8c0bf
summary:     A1

changeset:   0:5a118c651080
summary:     init

versus
cd B
hg log
changeset:   5:04157da570b3
summary:     Merged

changeset:   4:111f2448422b
summary:     A2

changeset:   3:3a8ecee8c0bf
summary:     A1

changeset:   2:e44ea38fbbb9
summary:     B2

changeset:   1:d2df37f438ac
summary:     B1

changeset:   0:5a118c651080
summary:     init


The other thing is that these numbers are going to be permanently
different for the two branches. Which means that you can't refer to
"revno 4 in branch A". Because someone with an effective clone of your
"branch A" might have different revisions.

We are working hard to decrease the cost of this ancestry lookup. We
have a couple plans which will allow us to determine the revision
numbers without having to look at all of history. We'll still look at
more than 0 history, but if we can bound the amount of history we have
to look at, it should help.


(I believe 'git log' defaults to showing the log based on a local sort
by date. Neither one tries to figure out that A1 and A2 were merged into
tip, which is another step that 'bzr log' does.)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH2jYBJdeBCYSNAAMRAh/KAJ9pK07hLXcDBZY+GqZUYWaemSXTTQCdE1U6
hO2ad7jQ6e4h+jqIgNsOYgM=
=CX95
-----END PGP SIGNATURE-----

Re: Emacs Bazaar repository

[Nicholas Allen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


| bzr clone http://bzr.notengoamigos.org/emacs/trunk/ emacs-bzr
|
| I ran it for 8 hours over a 2Mbit ADSL line then got bored (because it
| was crunching my hard disk) a killed it. The bzr status bar was around
| 70%.

This would probably be a lot faster over the smart server I guess -
hopefully comparable to git as the smart server should compress and send
the revisions in one go.

But I think this is a great example of where the idea of shallow
branches would be great. You would get your working copy (where you
could commit, status, merge, log, annotate etc) almost immediately and
Bazaar would download the remaining history in the background without
getting in your way. I hope the Bazaar team can implement something like
this fairly quickly as I think it is a killer feature and would
eliminate most performance problems when branching (especially for large
projects like Emacs).

I must say I am very impressed with Bazaar's developers - it's amazing
what they have done already and how quickly they develop new features.
So if anyone could pull something like this off I think it would be them ;-)

Cheers,

Nick
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH2kKC1+i51gqqEGkRAjjrAKC39OZkEPfI0793TTpDK9p4QwdhBwCgoar1
kgS/JJE7NI7TTpV8vzcv43M=
=QOoV
-----END PGP SIGNATURE-----

Re: Emacs Bazaar repository

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

On Fri, Mar 14, 2008 at 8:16 AM, Dan Nicolaescu <dann@ics.uci.edu> wrote:

>  The content is OK, but it takes 30 seconds to run
>  bzr log --limit=30 --short FILENAME

On Windows XP (with prebuilt binary of 1.2.0), the command

  bzr log --limit=30 --short lisp\ChangeLog

brought pagefile size up from ~500 MB (before running bazaar) to
>2.5GB (and I had to kill it). It is repeatable.

 Juanma

[-- Attachment #2: bzr1.jpg --]
[-- Type: image/jpeg, Size: 48335 bytes --]

Re: Emacs Bazaar repository

[Matthieu Moy]

David Kastrup <dak@gnu.org> writes:

> Andreas Schwab <schwab@suse.de> writes:
>
>> My first impression is that bzr is slow, so slow that it is completely
>> unusable.  How can it come that a simple bzr log takes more than a
>> minute to even start?  Even cvs log is instantaneous in comparison,
>> although it has to request the log from the server.
>
> I find this surprising: "git log" is pretty much instantaneous, and git
> recalculates a code piece's history in the process (renaming and copying
> is not tracked by git, but calculated on the fly when you ask for log
> output).

Actually the full "git log" can be long, but in practice, "git log"
seems _always_ instantaneous, since:

* git log pipes itself to less by default, so you get something stable
  in your terminal as soon as "git log" has outputed a few lines.

* git log can show the output as it finds it. It can show the output
  for HEAD immediately, then the ancestors of HEAD, then the
  grandparents, ...

As opposed to that, bzr has to get a global view of history at least
to get the revision numbers (there was some plans caching this
information, I don't know what's the status).

That said, the time for bzr log to start should clearly not be _that_
long. I suspect it's done on a light checkout (therefore needing
network access), which git can't do at all for example.

Re: Emacs Bazaar repository

[Matthew D. Fuller]

On Fri, Mar 14, 2008 at 10:18:23AM +0100 I heard the voice of
Juanma Barranquero, and lo! it spake thus:
> On Fri, Mar 14, 2008 at 8:16 AM, Dan Nicolaescu <dann@ics.uci.edu> wrote:
> 
> >  The content is OK, but it takes 30 seconds to run
> >  bzr log --limit=30 --short FILENAME
> 
> On Windows XP (with prebuilt binary of 1.2.0), the command
> 
>   bzr log --limit=30 --short lisp\ChangeLog
> 
> brought pagefile size up from ~500 MB (before running bazaar) to
> >2.5GB (and I had to kill it). It is repeatable.

Worth a note is that "bzr log $FILE" is currently _much_ slower and
heavier on memory than "bzr log" (unrestricted).

Re: Emacs Bazaar repository

[Juanma Barranquero]

On Fri, Mar 14, 2008 at 11:08 AM, Matthew D. Fuller
<fullermd@over-yonder.net> wrote:

>  Worth a note is that "bzr log $FILE" is currently _much_ slower and
>  heavier on memory than "bzr log" (unrestricted).

Yes. "bzr log" works fine.

Anyway, I'm not complaining, just reporting.

 Juanma

Re: Emacs Bazaar repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@suse.de>
>> Date: Thu, 13 Mar 2008 12:43:42 +0100
>> Cc: bazaar@lists.canonical.com, emacs-devel@gnu.org
>> 
>> Even cvs log is instantaneous in comparison,
>
> How do you mean ``instantaneous''?  For me, it takes 10 seconds to
> start displaying the log.

Which is ``instantaneous'' compared to the more than 60 seconds for bzr
log.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

"Jonathan Lange" <jml@mumak.net> writes:

> On Thu, Mar 13, 2008 at 8:58 PM, Andreas Schwab <schwab@suse.de> wrote:
>> "Jonathan Lange" <jml@mumak.net> writes:
>>
>>
>> > Actually, it's probably a good idea to do make a shared repository
>>  > called 'emacs-bzr' and use the original clone command inside that. So,
>>  >
>>  >   bzr init-repo emacs-bzr
>>  >   cd emacs-bzr
>>  >   bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>>  >
>>  > That way, Bazaar will share revisions between branches of Emacs,
>>  > making it faster to clone other branches.
>>
>>  Can you do that sharing also between different directories, that is the
>>  equivalent of git clone --reference?
>>
>
> I'm not exactly sure what you mean, since I don't know much about git.
> I'll tell you what I know, instead.

You can tell git to borrow objects from another cloned repository
(similar to a symbolic link), and this other repository can be located
anywhere in the filesystem.  AFAICS, for bzr to be able to share commits
both clones need to be located under a common, prepared directory, which
would not fit into my current way to handle repositories.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

"Jonathan Lange" <jml@mumak.net> writes:

> I was a bit curious about this so I asked my friendly neighbourhood
> Bazaar hacker (CCd). He tells me that comparing 'bzr log' to 'hg log'
> is like comparing apples with ... umm, apple trees. Bazaar's log
> displays all merges and thus has to do calculations on the whole
> ancestry graph.

As does git log, yet it is much faster.  Note that the bzr mirror of the
Emacs repo does not contain any merges.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

Forest Bond <forest@alittletooquiet.net> writes:

> Hi,
>
> On Thu, Mar 13, 2008 at 12:43:42PM +0100, Andreas Schwab wrote:
>> My first impression is that bzr is slow, so slow that it is completely
>> unusable.  How can it come that a simple bzr log takes more than a
>> minute to even start?  Even cvs log is instantaneous in comparison,
>> although it has to request the log from the server.
>
> You'll want to make sure you are using at least version 0.92;

An older version would not be able to use the repo at all.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

"Jonathan Lange" <jml@mumak.net> writes:

> As I mentioned in an earlier post, 'bzr log' and 'cvs log' are doing
> very different things. 'bzr log --line' is much closer to typical 'cvs
> log' behaviour.

That does not make any difference, still taking more than a minute to
start.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[David Kastrup]

Andreas Schwab <schwab@suse.de> writes:

> "Jonathan Lange" <jml@mumak.net> writes:
>
>> On Thu, Mar 13, 2008 at 8:58 PM, Andreas Schwab <schwab@suse.de> wrote:
>>>
>>>  Can you do that sharing also between different directories, that is
>>>  the equivalent of git clone --reference?
>>>
>>
>> I'm not exactly sure what you mean, since I don't know much about
>> git.  I'll tell you what I know, instead.
>
> You can tell git to borrow objects from another cloned repository
> (similar to a symbolic link), and this other repository can be located
> anywhere in the filesystem.  AFAICS, for bzr to be able to share
> commits both clones need to be located under a common, prepared
> directory, which would not fit into my current way to handle
> repositories.

"share commits" is somewhat misleading: clone --reference does not
actually share any structure between the repositories: they can be
completely unrelated.  What it does is, loosely speaking, to use one
repository as a compression dictionary for the other repository.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Emacs Bazaar repository

[Andreas Schwab]

"Jonathan Lange" <jml@mumak.net> writes:

> Also, there's now a patch for Bazaar to make 'bzr log --line' a lot
> faster -- http://bundlebuggy.aaronbentley.com/request/%3C47D9E60A.6030203%40canonical.com%3E

Will it be as fast as git log --pretty=oneline?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> That said, the time for bzr log to start should clearly not be _that_
> long. I suspect it's done on a light checkout (therefore needing
> network access), which git can't do at all for example.

There is definitely no network access involved, it is almost 100% CPU
time.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Thomas Lord]

Andreas Schwab wrote:
> An older version would not be able to use the repo at all.
>   


Perhaps that kind of consideration should be given some weight.

If "repo format" is not obviously pretty stable, how suitable
is it for an archival format?

Geeze, maybe Emacs *should* just use Arch.

-t

Re: Emacs Bazaar repository

[Matt Nordhoff]

Andreas Schwab wrote:
> My first impression is that bzr is slow, so slow that it is completely
> unusable.  How can it come that a simple bzr log takes more than a
> minute to even start?  Even cvs log is instantaneous in comparison,
> although it has to request the log from the server.

Yeah, the new pack disk format made many things faster, but it made
whole-history operations like "bzr log" slower, and the developers are
still working on that..

Re: Emacs Bazaar repository

[Matthieu Moy]

Andreas Schwab <schwab@suse.de> writes:

> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>
>> That said, the time for bzr log to start should clearly not be _that_
>> long. I suspect it's done on a light checkout (therefore needing
>> network access), which git can't do at all for example.
>
> There is definitely no network access involved, it is almost 100% CPU
> time.

Yes, right. Just reproduced here (perhaps with a faster machine than
yours) :

$ time bzr log | head -1
------------------------------------------------------------
bzr log  21.17s user 0.28s system 99% cpu 21.578 total
head -1  0.00s user 0.00s system 0% cpu 21.523 total
$ bzr --version
Bazaar (bzr) 1.3.0.dev.0
[...]
$ bzr info
Standalone tree (format: pack-0.92)
[...]

While on the git repo for Emacs,

$ time git log | head -1
commit 04eb7b6c65c8ec7550afb9cf317f51a1470f947c
git log  0.00s user 0.00s system 64% cpu 0.012 total
head -1  0.00s user 0.00s system 34% cpu 0.012 total

Similarly, I tested a commit touching a single file (echo foo >>
README), it takes 17 seconds with bzr, and 0.08 seconds with git.

Re: Emacs Bazaar repository

[Eli Zaretskii]

> From: Matthieu Moy <Matthieu.Moy@imag.fr>
> Date: Fri, 14 Mar 2008 10:58:13 +0100
> Cc: Andreas Schwab <schwab@suse.de>, emacs-devel@gnu.org,
> 	bazaar@lists.canonical.com
> 
> > Andreas Schwab <schwab@suse.de> writes:
> >
> >> My first impression is that bzr is slow, so slow that it is completely
> >> unusable.  How can it come that a simple bzr log takes more than a
> >> minute to even start?  Even cvs log is instantaneous in comparison,
> >> although it has to request the log from the server.
> [...]
> As opposed to that, bzr has to get a global view of history at least
> to get the revision numbers (there was some plans caching this
> information, I don't know what's the status).
> 
> That said, the time for bzr log to start should clearly not be _that_
> long.

Incidentally, why are we concentrating on "bzr log"? is that a
frequent operation?  With CVS, I find myself doing "cvs log" only once
in a few months, when I'm looking for a change corresponding to some
ChangeLog entry.

Aren't "push" and "pull" much more important, as far as speed is
concerned, for everyday work?  Also the equivalent of "cvs diff", I
think.  Those are the ops I use much more frequently than "log" and
"annotate".

Re: Emacs Bazaar repository

[David Ingamells]

Matthieu Moy wrote:
> Andreas Schwab <schwab@suse.de> writes:
>
>   
>> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>>
>>     
>>> That said, the time for bzr log to start should clearly not be _that_
>>> long. I suspect it's done on a light checkout (therefore needing
>>> network access), which git can't do at all for example.
>>>       
>> There is definitely no network access involved, it is almost 100% CPU
>> time.
>>     
>
> Yes, right. Just reproduced here (perhaps with a faster machine than
> yours) :
>
> $ time bzr log | head -1
> ------------------------------------------------------------
> bzr log  21.17s user 0.28s system 99% cpu 21.578 total
> head -1  0.00s user 0.00s system 0% cpu 21.523 total
> $ bzr --version
> Bazaar (bzr) 1.3.0.dev.0
> [...]
> $ bzr info
> Standalone tree (format: pack-0.92)
> [...]
>
> While on the git repo for Emacs,
>
> $ time git log | head -1
> commit 04eb7b6c65c8ec7550afb9cf317f51a1470f947c
> git log  0.00s user 0.00s system 64% cpu 0.012 total
> head -1  0.00s user 0.00s system 34% cpu 0.012 total
>
> Similarly, I tested a commit touching a single file (echo foo >>
> README), it takes 17 seconds with bzr, and 0.08 seconds with git.
>
>   
A small note of "warning" regarding such timing comparisons. make sure 
you are not comparing apples and oranges.

When we were choosing a new CMS tool I did a similar comparison between 
mercurial and bazaar, which mercurial won easily until I discovered why: 
mercurial first uses time stamps to check for potential updates - which 
leads to lost updates if the file update happens within one second of 
the checkout. bazaar is more thorough when checking for changes - this 
costs time but is much safer.

Re: Emacs Bazaar repository

[dhruva]

Hi,

On Fri, Mar 14, 2008 at 6:10 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
>  Incidentally, why are we concentrating on "bzr log"? is that a
>  frequent operation?  With CVS, I find myself doing "cvs log" only once
>  in a few months, when I'm looking for a change corresponding to some
>  ChangeLog entry.

Since I did the original posting with 'bzr log', I will post my reason.
I have been using Git which supports multiple branches in the same
folder. At times, I just want to make sure if I have all the latest
changes pulled in. Ideally, a pull would have done its job, just that
I try to make sure. I do that with a 'git log -l 10'. Since there are
so many branches, at times I would like to know the activity and try
building the one which has the most recent modification.
I agree that it is not the most important feature but a required feature.

Having tried a bunch of SCM, I must say GIT way of supporting multiple
branches under the same folder along with its speed it a sure winner.
I was opposing GIT due to its non-availability on M$, it is history
now and the port they have is really good. The build is streamlined on
M$ too, I pull their changes regularly, build, install and use the
bleeding edge. It has not failed me so far.
 If Mercurial had the ability to truly support multiple branches in
the same folder (with out requiring me to merge all branches before I
can pull - pull works only if there is a single tip/branch), I would
have preferred it mainly because it just needs PYTHON and nothing else
(GIT needs PERL and SHELL).
Speed wise, I find both GIT and Mercurial very close to each other.
GIT may be faster but not noticeable to human perceptions (well on
emacs repository I have tried). Here, I give GIT a thumbs up because
it has data for all branches and still the fastest. If mercurial
decides to support multiple branches under the same folder, I wonder
what impact it might have on the speed.

-dhruva

Re: Emacs Bazaar repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Incidentally, why are we concentrating on "bzr log"? is that a
> frequent operation?

IMHO it is.  It is the main source of revision information.  But it also
happend to be the first command that I tried after the clone.

> Aren't "push" and "pull" much more important, as far as speed is
> concerned, for everyday work?

Those are important, too, but I could not test them yet.

> Also the equivalent of "cvs diff", I think.  Those are the ops I use
> much more frequently than "log" and "annotate".

Yes, "diff" is quite important as well.  Unfortunately, bzr diff seems
to have the same performance problem.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

David Ingamells <david.ingamells@mapscape.eu> writes:

> When we were choosing a new CMS tool I did a similar comparison between
> mercurial and bazaar, which mercurial won easily until I discovered why:
> mercurial first uses time stamps to check for potential updates - which
> leads to lost updates if the file update happens within one second of the
> checkout. bazaar is more thorough when checking for changes - this costs
> time but is much safer.

But it shouldn't result in a 60 seconds delay.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Lennart Borgman (gmail)]

>  If Mercurial had the ability to truly support multiple branches in
> the same folder (with out requiring me to merge all branches before I
> can pull - pull works only if there is a single tip/branch), I would
> have preferred it mainly because it just needs PYTHON and nothing else
> (GIT needs PERL and SHELL).


How did they do that? It needs both perl and sh? Is there really any 
perl programmer who writes code that way?

Re: Emacs Bazaar repository

[dhruva]

Hi,

On Fri, Mar 14, 2008 at 6:40 PM, Lennart Borgman (gmail)
<lennart.borgman@gmail.com> wrote:
>  > have preferred it mainly because it just needs PYTHON and nothing else
>  > (GIT needs PERL and SHELL).
>
>
>  How did they do that? It needs both perl and sh? Is there really any
>  perl programmer who writes code that way?
>

PERL scripts in GIT 'bin' folder:
git-add--interactive
git-archimport
git-cvsexportcommit
git-cvsimport
git-cvsserver
git-instaweb
git-relink
git-remote
git-send-email
git-svn

SH scripts in GIT bin:
git-am
git-bisect
git-browse-help
git-checkout
git-citool
git-clone
git-filter-branch
git-gui
git-gui.tcl
git-instaweb
git-instaweb
git-lost-found
git-merge
git-merge-octopus
git-merge-one-file
git-merge-resolve
git-merge-stupid
git-mergetool
git-parse-remote
git-pull
git-quiltimport
git-rebase
git-rebase--interactive
git-repack
git-request-pull
git-sh-setup
git-stash
git-submodule
git-web--browse
gitk


-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Andreas Schwab]

dhruva <dhruvakm@gmail.com> writes:

> git-gui
> gitk

Those are actually tcl scripts.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Andreas Schwab]

John Arbash Meinel <john@arbash-meinel.com> writes:

> The other thing is that these numbers are going to be permanently
> different for the two branches. Which means that you can't refer to
> "revno 4 in branch A". Because someone with an effective clone of your
> "branch A" might have different revisions.

With git revisions are globally unique.  Pulling from a different
repository does not change revisions in any way.

> (I believe 'git log' defaults to showing the log based on a local sort
> by date. Neither one tries to figure out that A1 and A2 were merged into
> tip, which is another step that 'bzr log' does.)

I don't understand what "trying to figure out that ... were merged"
means.  If there is a merge commit git just shows it as such.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[David Kastrup]

John Arbash Meinel <john@arbash-meinel.com> writes:

> The biggest reason 'bzr log' is slow is because we spend some time
> analyzing the ancestry to give a "pretty" view, while git/hg do not.

git most certainly does.

> Specifically, when you do "bzr log" we traverse the ancestry to figure
> out when revisions were merged, etc.

What makes you think git doesn't?

> I believe plain "git log" just starts outputting the revisions as it
> encounters them, and "hg log" also outputs them as they are stored.

git has a large variety of options for selecting order and subset and
relation of what to output to the log.

It is still fast, even while doing rename/copying detection on the fly.

> (I believe 'git log' defaults to showing the log based on a local sort
> by date. Neither one tries to figure out that A1 and A2 were merged
> into tip, which is another step that 'bzr log' does.)

I suggest you actually check your beliefs against the actual program.
"The reason the other software is faster must be because it sucks in
comparison to ours." is a fallacy.  git has been developed by a set of
kernel-savvy developers working on a large code base with a necessity
for high speed (Linus merges several hundred patches from different
repositories daily).

Re: Emacs Bazaar repository

[James Westby]

On Fri, 2008-03-14 at 14:32 +0100, Andreas Schwab wrote:
> John Arbash Meinel <john@arbash-meinel.com> writes:
> 
> > The other thing is that these numbers are going to be permanently
> > different for the two branches. Which means that you can't refer to
> > "revno 4 in branch A". Because someone with an effective clone of your
> > "branch A" might have different revisions.
> 
> With git revisions are globally unique.  Pulling from a different
> repository does not change revisions in any way.

I think John intended to say

  "Because someone with an effective clone of your "branch A" might 
   have different revision numbers."

and as git doesn't use revision numbers this doesn't really apply.

bzr's revisions are also globally unique and immutable. It's just
their numbering that may change.

Thanks,

James

Re: Emacs Bazaar repository

[James Westby]

On Fri, 2008-03-14 at 14:35 +0100, David Kastrup wrote:
> John Arbash Meinel <john@arbash-meinel.com> writes:
> 
> > The biggest reason 'bzr log' is slow is because we spend some time
> > analyzing the ancestry to give a "pretty" view, while git/hg do not.
> 
> git most certainly does.

But it's analysis is different. It is doing something that has fewer 
constraints on the output that bzr log.

You still end up with a list of revisions, but the ordering on bzr's
is more complex to calculate.

> 
> > Specifically, when you do "bzr log" we traverse the ancestry to figure
> > out when revisions were merged, etc.
> 
> What makes you think git doesn't?
> 

I don't think John articulated his point as he would have liked there.

> > I believe plain "git log" just starts outputting the revisions as it
> > encounters them, and "hg log" also outputs them as they are stored.
> 
> git has a large variety of options for selecting order and subset and
> relation of what to output to the log.
> 

However it doesn't have one that outputs them in the same order as bzr.

> It is still fast, even while doing rename/copying detection on the fly.
> 

It is fast, no-one is disagreeing with that.

> > (I believe 'git log' defaults to showing the log based on a local sort
> > by date. Neither one tries to figure out that A1 and A2 were merged
> > into tip, which is another step that 'bzr log' does.)
> 
> I suggest you actually check your beliefs against the actual program.
> "The reason the other software is faster must be because it sucks in
> comparison to ours." is a fallacy.  git has been developed by a set of
> kernel-savvy developers working on a large code base with a necessity
> for high speed (Linus merges several hundred patches from different
> repositories daily).
> 

I'm sure that John did not intend to say that git "sucks", and I firmly
believe that none of us believes that, and certainly no-one would
argue that it is not faster than bzr.

I think there are still criticisms of the UI, even though it has
significantly improved recently.

Thanks,

James

Re: Emacs Bazaar repository

[David Kastrup]

James Westby <jw+debian@jameswestby.net> writes:

> On Fri, 2008-03-14 at 14:35 +0100, David Kastrup wrote:
>> John Arbash Meinel <john@arbash-meinel.com> writes:
>> 
>> > The biggest reason 'bzr log' is slow is because we spend some time
>> > analyzing the ancestry to give a "pretty" view, while git/hg do not.
>> 
>> git most certainly does.
>
> But it's analysis is different.

How?

> It is doing something that has fewer constraints on the output that
> bzr log.
>
> You still end up with a list of revisions, but the ordering on bzr's
> is more complex to calculate.

Then it should improve its data structures.

>> > Specifically, when you do "bzr log" we traverse the ancestry to figure
>> > out when revisions were merged, etc.
>> 
>> What makes you think git doesn't?
>
> I don't think John articulated his point as he would have liked there.
>
>> > I believe plain "git log" just starts outputting the revisions as it
>> > encounters them, and "hg log" also outputs them as they are stored.
>> 
>> git has a large variety of options for selecting order and subset and
>> relation of what to output to the log.
>
> However it doesn't have one that outputs them in the same order as
> bzr.

The default is topological order.  I don't see anything that bzr would
offer additionally.

> I'm sure that John did not intend to say that git "sucks", and I
> firmly believe that none of us believes that, and certainly no-one
> would argue that it is not faster than bzr.
>
> I think there are still criticisms of the UI, even though it has
> significantly improved recently.

Sigh.  So it is fine if the logs of bzr are really slow because the UI
of git is considered not all too hot.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Emacs Bazaar repository

[James Westby]

On Fri, 2008-03-14 at 14:59 +0100, David Kastrup wrote:
> James Westby <jw+debian@jameswestby.net> writes:
> 
> > On Fri, 2008-03-14 at 14:35 +0100, David Kastrup wrote:
> >> John Arbash Meinel <john@arbash-meinel.com> writes:
> >> 
> >> > The biggest reason 'bzr log' is slow is because we spend some time
> >> > analyzing the ancestry to give a "pretty" view, while git/hg do not.
> >> 
> >> git most certainly does.
> >
> > But it's analysis is different.
> 
> How?

May I point you to my blog post of yesterday that explains some of
these issues, rather than spell if all out here?

http://jameswestby.net/weblog/bzr/01-revision-numbers.html

The last section is on this particular issue.

The gist of it is that topological sorting means that children appear
before their parents. bzr does "merge sorting" that ensures that a
revision does not appear after a revision that it is not an ancestor
of.

> > I'm sure that John did not intend to say that git "sucks", and I
> > firmly believe that none of us believes that, and certainly no-one
> > would argue that it is not faster than bzr.
> >
> > I think there are still criticisms of the UI, even though it has
> > significantly improved recently.
> 
> Sigh.  So it is fine if the logs of bzr are really slow because the UI
> of git is considered not all too hot.
> 

Please don't put words in to my mouth.

I was trying to say that while we might acknowledge that git is faster,
there are still things we are trying to do with bzr that make it
worthwile working on it, rather than all switching to hack on git.

While we may be able to offer some UI improvements to git itself there
may be fundamental differences that mean git may always fall short of
what we would like. There are also things that bzr can do that git
will never do (barring any big changes in direction), for instance
foreign branch support.

Thanks,

James

Re: Emacs Bazaar repository

[Matthieu Moy]

Andreas Schwab <schwab@suse.de> writes:

> I don't understand what "trying to figure out that ... were merged"
> means.  If there is a merge commit git just shows it as such.

"git log" gives you a flat view. When it encounters a merge commit, it
displays it, and the shows you the two (or more) branches merged by
that commit more or less concatenated together.

IOW, a revision that were brought in your branch by a merge and a
revision that you commited by yourself in the branch is shown in the
same way in git.

As opposed to that, bzr makes the distinction between "mainline
revisions" (that you commited in the branch), and merged revisions
(ancestors of merge commits that you brought here with a merge).
Merged revisions are shown indented below the merge commit which
brought them here.

That's a real difference in the way git and bzr deal with the history
DAG, not just about "log". For example, git merge defaults to
fast-forward (i.e. if you merge a revision you are a direct ancestor
of, you just jump to that revision without creating a new commit).

Re: Emacs Bazaar repository

[Matthieu Moy]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:

>>  If Mercurial had the ability to truly support multiple branches in
>> the same folder (with out requiring me to merge all branches before I
>> can pull - pull works only if there is a single tip/branch), I would
>> have preferred it mainly because it just needs PYTHON and nothing else
>> (GIT needs PERL and SHELL).
>
> How did they do that? It needs both perl and sh? Is there really any
> perl programmer who writes code that way?

The core git is in C, and designed to be used in scripts.

Then, UI commands have usually been prototyped in shell-script, but
there's an ongoing effort to re-write them in C (mostly because
shell-scripts sucks when it comes to robustness and portability).

Some commands have been written in perl instead of C or shell,
probably because they have been written by people who like perl.

Re: Emacs Bazaar repository

[Matthieu Moy]

Andreas Schwab <schwab@suse.de> writes:

> Yes, "diff" is quite important as well.  Unfortunately, bzr diff seems
> to have the same performance problem.

It shouldn't be on a "normal" setup at least with a recent enough bzr.
Git is obviously faster, but for example, on an emacs tree, "bzr diff"
takes 0.3 seconds, while Git takes 0.05. So, the time taken by bzr is
acceptable to me.

Re: Emacs Bazaar repository

[Nicholas Allen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
I personally love the format Bazaar uses and revision numbers as well as 
ids. It is IMHO better than git's. However, it should not imply such a 
large performance penalty. I'm sure with better algorithms and data 
structures this problem can be solved.

There must be a lot of information that can be computed from the 
revision data and cached which is in a format better suited for certain 
operations. When I branch I don't want to download this calculated data 
as I can compute it in the background on my own machine a lot faster 
than I can download it.

So it seems that the packs format is not optimal for logs on single 
files. Can't this be represented by a computed and cached alternative 
format? At least the next time I log a file it will be very fast. Bazaar 
could also do this before I even ask for a log so it is ready the first 
time.

Just ideas - I don't know if it makes sense implementation wise....

Nick


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iD8DBQFH2ovGbpmWsXfOU58RAkqPAKCV247b0vgev7/vsEFumZhQsSmePQCguhLR
uR/WfFf6TvWAY0510fbIuUM=
=v580
-----END PGP SIGNATURE-----

Re: Emacs Bazaar repository

[Lennart Borgman (gmail)]

Matthieu Moy wrote:
> "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:
> 
>>>  If Mercurial had the ability to truly support multiple branches in
>>> the same folder (with out requiring me to merge all branches before I
>>> can pull - pull works only if there is a single tip/branch), I would
>>> have preferred it mainly because it just needs PYTHON and nothing else
>>> (GIT needs PERL and SHELL).
>> How did they do that? It needs both perl and sh? Is there really any
>> perl programmer who writes code that way?
> 
> The core git is in C, and designed to be used in scripts.
> 
> Then, UI commands have usually been prototyped in shell-script, but
> there's an ongoing effort to re-write them in C (mostly because
> shell-scripts sucks when it comes to robustness and portability).
> 
> Some commands have been written in perl instead of C or shell,
> probably because they have been written by people who like perl.

Thanks Matthieu, for the explanation. My point was also that perl is far 
more portable then sh (beside beeing more powerful as a scripting language).

Re: Emacs Bazaar repository

[Andreas Schwab]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> As opposed to that, bzr makes the distinction between "mainline
> revisions" (that you commited in the branch), and merged revisions
> (ancestors of merge commits that you brought here with a merge).

Is that a useful distinction?  I think git treats all branches equal,
there is no "mainline".  If you merge two branches that touch different
parts of the tree there is no need to distinguish the two threads.

> That's a real difference in the way git and bzr deal with the history
> DAG, not just about "log". For example, git merge defaults to
> fast-forward (i.e. if you merge a revision you are a direct ancestor
> of, you just jump to that revision without creating a new commit).

If you just move forward in the history it is not really a merge, so
there should not be a need for creating a marker for it.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Nicholas Allen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
Andreas Schwab wrote:
| Matthieu Moy <Matthieu.Moy@imag.fr> writes:
|
|> As opposed to that, bzr makes the distinction between "mainline
|> revisions" (that you commited in the branch), and merged revisions
|> (ancestors of merge commits that you brought here with a merge).
|
| Is that a useful distinction?  I think git treats all branches equal,
| there is no "mainline".  If you merge two branches that touch different
| parts of the tree there is no need to distinguish the two threads.
I disagree! I LOVE this distinction. Each commit on a mainline is for a 
feature if you use feature branches. I want to see these 2 features as 
separate things regardless of whether they touched the same or different 
parts of the tree. Bazaar has the UI spot on I think - it just needs to 
perform better. I have been using Bazaar since 0.8 and I can tell you 
the performance improvements they have made so far are huge. They are 
still working on performance and I'm sure they will fix this issue in 
the not too distant future as they have a very good track record of 
doing so....

Cheers,

Nick
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iD8DBQFH2o2XbpmWsXfOU58RApebAJsG7OWmyRf6triVXQ9nafglYd60rQCcC3Xj
0CHjLN+GLCvKOfrnZAZspIw=
=VYHA
-----END PGP SIGNATURE-----

Re: Emacs Bazaar repository

[Andreas Schwab]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> Andreas Schwab <schwab@suse.de> writes:
>
>> Yes, "diff" is quite important as well.  Unfortunately, bzr diff seems
>> to have the same performance problem.
>
> It shouldn't be on a "normal" setup at least with a recent enough bzr.
> Git is obviously faster, but for example, on an emacs tree, "bzr diff"
> takes 0.3 seconds, while Git takes 0.05. So, the time taken by bzr is
> acceptable to me.

What is the fastest way to get the difference of a revision relative to
its ancestor?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Jelmer Vernooij]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

Am Freitag, den 14.03.2008, 15:41 +0100 schrieb Andreas Schwab:
> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
> 
> > Andreas Schwab <schwab@suse.de> writes:
> >
> >> Yes, "diff" is quite important as well.  Unfortunately, bzr diff seems
> >> to have the same performance problem.
> >
> > It shouldn't be on a "normal" setup at least with a recent enough bzr.
> > Git is obviously faster, but for example, on an emacs tree, "bzr diff"
> > takes 0.3 seconds, while Git takes 0.05. So, the time taken by bzr is
> > acceptable to me.
> 
> What is the fastest way to get the difference of a revision relative to
> its ancestor?
if you mean its left hand side ancestor: 

bzr diff -c <rev>

Cheers,

Jelmer
-- 
Jelmer Vernooij <jelmer@samba.org> - http://samba.org/~jelmer/
Jabber: jelmer@jabber.fsfe.org

[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 307 bytes --]

Re: Emacs Bazaar repository

[Giorgos Keramidas]

On 2008-03-14 10:46, Jonathan Lange <jml@mumak.net> wrote:
> On Thu, Mar 13, 2008 at 7:04 PM, dhruva <dhruvakm@gmail.com> wrote:
> > Hi,
> >   I got bzr setup and running on my M$-XP box. I got the bzr repo of
> >  emacs (by downloading the emacs.tar.gz) and updated it to the latest.
> >   I wanted to make sure the recent changesets are pulled in. I typed
> >  "bzr log -l 10". It took ~24 seconds (elapsed time from perfmon)
> >  consistently. Whereas Mercurial (hg log -l 10) on the trunk to 2
> >  seconds (and the mercurial repo is on a CIFS mounted drive while bzr
> >  repo was on local disk). GIT was fastest though even on a CIFS mounted
> >  system.
> >
>
> I was a bit curious about this so I asked my friendly neighbourhood
> Bazaar hacker (CCd). He tells me that comparing 'bzr log' to 'hg log'
> is like comparing apples with ... umm, apple trees. Bazaar's log
> displays all merges and thus has to do calculations on the whole
> ancestry graph. Mercurial's log just displays the mainline history.
> 'bzr log --line' is a better comparison.

The `glog' extension of Mercurial displays an ancestry graph and it can
show the last 3 changesets in less than 1/3 of a second, on a system
that is currently building the CVS 'HEAD' of FreeBSD and the CVS 'HEAD'
of Emacs itself:

  keramida@kobe:/hg/emacs/gker$ /usr/bin/time hg glog -l3
  @    changeset:   1988:601be9d00890
  |\   branch:      keramida
  | |  tag:         tip
  | |  parent:      1724:25ffb390e254
  | |  parent:      1987:c2bdce21cc28
  | |  user:        Giorgos Keramidas <keramida@ceid.upatras.gr>
  | |  date:        Fri Mar 14 16:48:24 2008 +0200
  | |  summary:     Merge from HEAD
  | |
  | o  changeset:   1987:c2bdce21cc28
  | |  branch:      HEAD
  | |  user:        bastien1
  | |  date:        Fri Mar 14 10:13:27 2008 +0000
  | |  summary:     * textmodes/flyspell.el (nxml-mode): Add the right.
  | |
  | o  changeset:   1986:414912c00a1f
  | |  branch:      HEAD
  | |  user:        miles
  | |  date:        Fri Mar 14 10:06:41 2008 +0000
  | |  summary:     Add arch tagline
  | |
          0.25 real         0.15 user         0.08 sys
  keramida@kobe:/hg/emacs/gker$

I'm not sure if `wait, because we are calculating the changeset
ancestry' is a valid reason for taking too long to show the last two or
three log entries...

> Definitely. I hate waiting for anything, computers most of all. I'll
> hassle Martin and try to get this fixed. Alternatively, the Bazaar
> project welcomes patches with open arms. :)

That's cool.  If Emacs converts to bzr, we need all the speed we can
get.  The CVS repository of Emacs is one of the CVS repositories with
the longest uninterrupted history I have ever seen :)

Re: Emacs Bazaar repository

[Andreas Schwab]

Jelmer Vernooij <jelmer@samba.org> writes:

> Am Freitag, den 14.03.2008, 15:41 +0100 schrieb Andreas Schwab:
>> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>> 
>> > Andreas Schwab <schwab@suse.de> writes:
>> >
>> >> Yes, "diff" is quite important as well.  Unfortunately, bzr diff seems
>> >> to have the same performance problem.
>> >
>> > It shouldn't be on a "normal" setup at least with a recent enough bzr.
>> > Git is obviously faster, but for example, on an emacs tree, "bzr diff"
>> > takes 0.3 seconds, while Git takes 0.05. So, the time taken by bzr is
>> > acceptable to me.
>> 
>> What is the fastest way to get the difference of a revision relative to
>> its ancestor?
> if you mean its left hand side ancestor: 

The test repo we are looking at does not have any merges yet.  How would
a display of a merge commit look like?

> bzr diff -c <rev>

This is what I was refering to above.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[David Allouche]

On Fri, Mar 14, 2008 at 3:30 PM, Andreas Schwab <schwab@suse.de> wrote:
> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>
>  > As opposed to that, bzr makes the distinction between "mainline
>  > revisions" (that you commited in the branch), and merged revisions
>  > (ancestors of merge commits that you brought here with a merge).
>
>  Is that a useful distinction?  I think git treats all branches equal,
>  there is no "mainline".

Bzr also treats all branches as equal. There is no "mainline branch"
the tool knows of. But there are "mainline revisions" in the ancestry
of a given revision (or branch tip).

>  If you merge two branches that touch different
>  parts of the tree there is no need to distinguish the two threads.

In bzr, the mainline ancestry of a revision is the transitive closure
of the leftmost parent. The mainline ancestry is considered important
because it records the sequence of commits that occured on a given
branch.

It gives a better view of the intent of the committer. For example, a
revision with the message "merge from trunk" usually means "take in
whatever got in the trunk", not "merge feature a, bugfix b, cleanup c,
refactoring d, and 43 other equally crucial changes". Most merges do
not reflect essential work that occured on the branch.

On the other hand, the mainline commits usually capture the essential
work that occured on a branch. By distinguishing them, bzr makes it
easier to read the history of a branch.

>  > That's a real difference in the way git and bzr deal with the history
>  > DAG, not just about "log". For example, git merge defaults to
>  > fast-forward (i.e. if you merge a revision you are a direct ancestor
>  > of, you just jump to that revision without creating a new commit).
>
>  If you just move forward in the history it is not really a merge, so
>  there should not be a need for creating a marker for it.

You can do fast forward in bzr with "pull". Merge behave differently
because we consider that "merge" _as a task_ has a different meaning
_to the user_ than pull.

Typically, a gatekeeper only merges, then run the test suite, and only
commit if the test suite pass. The gatekeeper's branch mainline
history will only contain revisions that passed the test suite.

The way you make it sound, the git approach is "if we can get the same
result by fast forward, we do not need a new commit". The bzr approach
is that a new commit provides important historical information
regardless.

Re: Emacs Bazaar repository

[James Westby]

On Fri, 2008-03-14 at 16:15 +0100, David Allouche wrote:
> You can do fast forward in bzr with "pull". Merge behave differently
> because we consider that "merge" _as a task_ has a different meaning
> _to the user_ than pull.

And if you disagree please add a bzr alias to alias "merge" to
"merge --pull", and you will get the behaviour that you want.

Thanks,

James

Re: Emacs Bazaar repository

[Matthieu Moy]

Andreas Schwab <schwab@suse.de> writes:

> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>
>> As opposed to that, bzr makes the distinction between "mainline
>> revisions" (that you commited in the branch), and merged revisions
>> (ancestors of merge commits that you brought here with a merge).
>
> Is that a useful distinction?

It really depends on your flow.

If the organization of developers forms a kind of tree, then on each
branch, bzr will show the revisions in a kind of hierarchical way,
like

42: Merged feature foo
    41.3: bugfix in implementation of foo
    41.2: added NEWS entry
    41.1: implement foo
41: Merged feature bar
    ...
    ...

So, you can get a global view of history following mainline revisions,
and go into details when walking through merged revisions. The bzr
people usually like this distinction.

There are drawbacks though. One of them is that this pushes people
towards a flow where the mainline is different from other branches. If
you don't, then the hierarchical view of bzr will look like garbage,
you'll see some "keep in sync with upstream" commits in the middle of
mainline, ... People like Torvalds find this unacceptable.

So, asking whether bzr does the right thing is like asking whether
Emacs is better than vim. One is obviously better than the other, and
everybody thinking a different way has to be plain stupid, but people
still didn't agree on which is which ;-).

Re: Emacs Bazaar repository

[David Kastrup]

Nicholas Allen <allen@ableton.com> writes:

> Bazaar has the UI spot on I think - it just needs to perform better. I
> have been using Bazaar since 0.8 and I can tell you the performance
> improvements they have made so far are huge. They are still working on
> performance and I'm sure they will fix this issue in the not too
> distant future as they have a very good track record of doing so....

Reminds me of a job I once did in geodesy.  The previous people working
on the job (partly doctors of informatics) had done several iterations
of optimizations on the code, halving its running time in the process,
then squeezing off some more.  Problem was that the data sets to be
processed were expected to more than triple, and the stuff was already
running a day or so.

I tried working myself into the code (I always have a hard time
understanding code of others, and one of the reasons I was put on the
job was that the code was getting more fragile) and after about two
weeks gave up.  I just don't have the mental capacity to find myself
comfortable around in a mess I did not create myself.  So I just spent
two days on thinking about an algorithm and optimizations and stuff and
implemented it from scratch.  Took probably another two weeks of coding
and testing where it wasn't dumping core or running into loops or
similar.  And then I got stuck debugging it.  The stuff just terminated
after a few minutes of run time, and I was running out of ideas what was
wrong.

After about a day of placing breakpoints and looking at the data
structures I was running out of ideas and turned to looking at the
generated output.

It was correct.  Since that time I have a healthy dose of scepticism
about people (and teams) tinkering with their algorithms and "coming
through".

The real metric is not how much you improve things, but where they
should be in the first place.

I am not implying that this is happening here.  But if we are several
orders of magnitude behind the competition, the proven ability of
squeezing off some runtime at the cost of legibility (and thus also the
viability of further optimizations without destabilization) is not a
useful metric.

Re: Emacs Bazaar repository

[Daniel Mark Watkins]

[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]

Hi David,

On Fri, 14 Mar 2008 17:17:01 +0100
David Kastrup <dak@gnu.org> wrote:
> It was correct.  Since that time I have a healthy dose of scepticism
> about people (and teams) tinkering with their algorithms and "coming
> through".
> 
> The real metric is not how much you improve things, but where they
> should be in the first place.
> 
> I am not implying that this is happening here.  But if we are several
> orders of magnitude behind the competition, the proven ability of
> squeezing off some runtime at the cost of legibility (and thus also
> the viability of further optimizations without destabilization) is
> not a useful metric.
The impression I get is that Bazaar has its concerns well-separated
enough internally that an entirely new algorithm for a given part of
the code base can be substituted in without a great deal of hassle.
That is to say, many of the performance improvements being made are
not tweaks to existing algorithms, but replacements of the existing
algorithms with better ones. A recent example of this would be knits
being replaced by packs in the storage layer.


Dan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Dan Nicolaescu]

Eli Zaretskii <eliz@gnu.org> writes:

  > > From: Matthieu Moy <Matthieu.Moy@imag.fr>
  > > Date: Fri, 14 Mar 2008 10:58:13 +0100
  > > Cc: Andreas Schwab <schwab@suse.de>, emacs-devel@gnu.org,
  > > 	bazaar@lists.canonical.com
  > > 
  > > > Andreas Schwab <schwab@suse.de> writes:
  > > >
  > > >> My first impression is that bzr is slow, so slow that it is completely
  > > >> unusable.  How can it come that a simple bzr log takes more than a
  > > >> minute to even start?  Even cvs log is instantaneous in comparison,
  > > >> although it has to request the log from the server.
  > > [...]
  > > As opposed to that, bzr has to get a global view of history at least
  > > to get the revision numbers (there was some plans caching this
  > > information, I don't know what's the status).
  > > 
  > > That said, the time for bzr log to start should clearly not be _that_
  > > long.
  > 
  > Incidentally, why are we concentrating on "bzr log"? is that a
  > frequent operation?  

For me it is.  Every time I change some code that I am not 100% sure I
know, I do a vc-annotate and then from the *vc-annotate* buffer vc-log
and vc-diff on the "interesting" lines.

  > Aren't "push" and "pull" much more important, as far as speed is
  > concerned, for everyday work?  Also the equivalent of "cvs diff", I
  > think.  Those are the ops I use much more frequently than "log" and
  > "annotate".

diff is surely much more frequent than "log" and "annotate", but it also
takes more than a minute with bzr (much slower than CVS).

Re: Emacs Bazaar repository

[Stefan Monnier]

> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:

> bzr clone http://bzr.notengoamigos.org/emacs/trunk/

Other than issues of performance, one of the missing elements in this
repository is all the merge information.

Most of the merges we've done have been done via Arch.  IIUC people have
been able to retrofit this info into the Git tree, and I think it'd be
very worthwhile to retrofit it into the Bzr tree.

How can we add the merge info into the above Bzr tree?


        Stefan

Re: Emacs Bazaar repository

[Goffredo Baroncelli]

I Jason,

I setup my webserve interface on a clone of your repo. You can found it at

http://goffredo-baroncelli.homelinux.net/bazaar-dev/emacs.trunk

I am working on a speed improvement (caching the revisionid), in order to 
minimize the delay (now the system requires 20 seconds in order to give a 
page).

BR
Goffredo

Re: Emacs Bazaar repository

[Jason Earl]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> After a few false starts I have been successful (at least as far as I
>> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
>> is interested in playing with the results you can check out the HEAD of
>> the CVS repository with:
>
>> bzr clone http://bzr.notengoamigos.org/emacs/trunk/
>
> Other than issues of performance, one of the missing elements in this
> repository is all the merge information.

Yeah, I didn't notice that until people started talking about bzr log
and I realized that it wasn't showing the merge information.

> Most of the merges we've done have been done via Arch.  IIUC people
> have been able to retrofit this info into the Git tree, and I think
> it'd be very worthwhile to retrofit it into the Bzr tree.

If someone knows how the folks that made the git repository did this
maybe I could work something out.  Or possibly I could take another
whack at creating a bzr repository from the git repository instead of
from CVS.

Alternatively, I could experiment with creating a bzr repository from
arch.  Where are those branches located?

> How can we add the merge info into the above Bzr tree?

I've certainly got some things I can try.

Jason

Re: Emacs Bazaar repository

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> How can we add the merge info into the above Bzr tree?

Probably the easiest way is by starting from
<http://repo.or.cz/w/emacs.git>.  I have spend quite some time to add
merge commits to this tree or updating the automatically created ones to
make them most useful.  I have also corrected a few mistakes that crept
in due to shortcomings of cvsps or misuses of cvs.

It should be possible to losslessly convert this tree to whatever VCS we
end up using.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Stefan Monnier]

>> How can we add the merge info into the above Bzr tree?

> Probably the easiest way is by starting from
> <http://repo.or.cz/w/emacs.git>.  I have spend quite some time to add
> merge commits to this tree or updating the automatically created ones to
> make them most useful.  I have also corrected a few mistakes that crept
> in due to shortcomings of cvsps or misuses of cvs.

> It should be possible to losslessly convert this tree to whatever VCS we
> end up using.

Is this git tree "better" than the one in git.sv.gnu.org?
If so, can someone update the one in git.sv.gnu.org to include the
improvements that are in http://repo.or.cz/w/emacs.git.

Even if we end up using this tree to generate the initial Bzr tree, I'd
be interested to know who to add merge info to a Bzr repository.


        Stefan

Re: Emacs Bazaar repository

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> How can we add the merge info into the above Bzr tree?
>
>> Probably the easiest way is by starting from
>> <http://repo.or.cz/w/emacs.git>.  I have spend quite some time to add
>> merge commits to this tree or updating the automatically created ones to
>> make them most useful.  I have also corrected a few mistakes that crept
>> in due to shortcomings of cvsps or misuses of cvs.
>
>> It should be possible to losslessly convert this tree to whatever VCS we
>> end up using.
>
> Is this git tree "better" than the one in git.sv.gnu.org?

Yes, IMVHO.  To get a picture of what is different you can run "gitk
--all" in both trees.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Stefan Monnier]

>> Aren't "push" and "pull" much more important, as far as speed is
>> concerned, for everyday work?  Also the equivalent of "cvs diff", I
>> think.  Those are the ops I use much more frequently than "log" and
>> "annotate".

> diff is surely much more frequent than "log" and "annotate", but it also
> takes more than a minute with bzr (much slower than CVS).

bzr diff <somefile> is pretty much instantaneous for me.
Even on the whole tree, it's just a couple seconds.  Good enough for me.
OTOH vc-diff on a bzr file takes ages (like 10s).  Not sure yet what's
going on.


        Stefan

Re: Emacs Bazaar repository

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> bzr diff <somefile> is pretty much instantaneous for me.
> Even on the whole tree, it's just a couple seconds.  Good enough for me.
> OTOH vc-diff on a bzr file takes ages (like 10s).  Not sure yet what's
> going on.

As soon as you start using revision numbers bzr becomes slow.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Mathias Dahl]

>  As soon as you start using revision numbers bzr becomes slow.

Do you mean the revision *numbers* per se, or the fact that there are
more revisions? Quite strange behavior for a VCS, the whole point is
to track different versions/revisions of files...

Re: Emacs Bazaar repository

[Andreas Schwab]

"Mathias Dahl" <mathias.dahl@gmail.com> writes:

>>  As soon as you start using revision numbers bzr becomes slow.
>
> Do you mean the revision *numbers* per se, or the fact that there are
> more revisions?

The problem seems to be connected to the way bzr computes revision
numbers from the tree.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Thomas Lord]

Stefan Monnier wrote:
> Even if we end up using this tree to generate the initial Bzr tree, I'd
> be interested to know who to add merge info to a Bzr repository.
>
>   


This is a frustrating development, from my perspective.

Arch was designed to *not* be monolithic, in some specific
ways that are relevant to problems like this:

Arch defines a "global name-space" for projects, branches,
and revisions -- but the idea is that you can use that name-space
without using all of Arch.   Users disliked a few superficial
attributes of the Arch name-space but the basic idea of
having one is sound.   Without picking a specific DVCS,
one can still pick a distributed, global name-space for revisions.

Arch records merge history in the source tree itself, using
ordinary files, referring to that global name-space.   The
source tree, not some external database, tells you what has
been merged into a given revision.  So, again,
you can use Arch's way of recording merge history even if
you don't use Arch itself.   In practice, it's like having a
Changelog but an extra-fancy changelog that tells you
(or the merge tools you use) the merge history.   All that the
underlying DVCS has to do is make it possible to look up
various trees (or deltas between them) using the global names.

If you start with those two things, global names for
revisions and in-tree merge-history, then your perspective
on distributed revision control should change completely.

For example, merge operations need not any longer be
some built-in facility of the revision control system.  Rather,
merge operations are, by definition, algorithms that do some
diffing and patching by looking at in-tree merge history
and comparing various trees that are named there.   A merge
operator can be coded abstractly, in a way that works
across multiple revision control systems.

If a project like emacs, instead of picking a DVCS, picks
a naming convention for revisions and a representation
convention for merge history, then portability between
DVCS systems becomes a largely moot issue.

-t

Re: Emacs Bazaar repository

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
> Incidentally, why are we concentrating on "bzr log"? is that a
> frequent operation?  With CVS, I find myself doing "cvs log" only once
> in a few months, when I'm looking for a change corresponding to some
> ChangeLog entry.

All of this is anecdotal, but for what it's worth, I find myself using
'cvs log' on the Emacs tree more like once a week (and more often than
that, when I'm actively working on something).

The lesson here may be that developers' habits can differ greatly!

-Karl

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

Thomas Lord writes:
 > Andreas Schwab wrote:
 > > An older version would not be able to use the repo at all.
 > >   
 > 
 > 
 > Perhaps that kind of consideration should be given some weight.
 > 
 > If "repo format" is not obviously pretty stable, how suitable
 > is it for an archival format?

As long as new version can read old repos reliably, new repos can
represent a superset of the old metadata, and there's a conversion
utility, this is not a problem.  Generic database managers deal with
these problems all the time, it's "solved" in some sense.

Question in my mind is, given bzr's propensity to change
representations, does the bzr development team have somebody who's
worked on that kind of function, and demands that functionality be
rock-solid?

Re: Emacs Bazaar repository

[Martin Geisler]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

dhruva <dhruvakm@gmail.com> writes:

> Having tried a bunch of SCM, I must say GIT way of supporting multiple
> branches under the same folder along with its speed it a sure winner.
> I was opposing GIT due to its non-availability on M$, it is history
> now and the port they have is really good. The build is streamlined on
> M$ too, I pull their changes regularly, build, install and use the
> bleeding edge. It has not failed me so far.
>
>  If Mercurial had the ability to truly support multiple branches in
> the same folder (with out requiring me to merge all branches before I
> can pull - pull works only if there is a single tip/branch), I would
> have preferred it mainly because it just needs PYTHON and nothing else
> (GIT needs PERL and SHELL).

I think you are confusing several 'heads' with several 'branches' here.
And even if you have several heads in your Mercurial repository, you can
certainly still pull in new changesets.

> Speed wise, I find both GIT and Mercurial very close to each other.
> GIT may be faster but not noticeable to human perceptions (well on
> emacs repository I have tried). Here, I give GIT a thumbs up because
> it has data for all branches and still the fastest. If mercurial
> decides to support multiple branches under the same folder, I wonder
> what impact it might have on the speed.

Have you searched on the Mercurial Wiki? You have both 'local branches'
and 'named branches' to choose from:

  http://www.selenic.com/mercurial/wiki/index.cgi/LocalbranchExtension
  http://www.selenic.com/mercurial/wiki/index.cgi/NamedBranches

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multi-Party Computation) to Python. See: http://viff.dk/.

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

John Arbash Meinel writes:

 > (I believe 'git log' defaults to showing the log based on a local sort
 > by date. Neither one tries to figure out that A1 and A2 were merged into
 > tip, which is another step that 'bzr log' does.)

YAGNI most of the time, though.  John, I know where you're coming
from, but it looks like you're throwing up red herrings and making
excuses.  There's no question that git is good enough for Emacs on a
day-to-day basis.  bzr's performance problems make me wonder if it is.

It would be interesting to see a "time git rev-list --topo-order"
(which does the same thing as "bzr log" AFAICS) on the relevant repo.
Here are two timings on my most complicated XEmacs git repo (but it
only has 601 changesets, including 40 branches and 38 merges):

chibi:git-staging steve$ time git-rev-list --topo-order HEAD
[[ output omitted ]]

real    0m1.131s	user    0m0.060s	sys     0m0.124s

chibi:git-staging steve$ time git-rev-list --topo-order HEAD | wc
     601     601   24641

real    0m0.423s	user    0m0.060s	sys     0m0.130s

So let's scale that up to 90,000 commits assuming O(n) (which may not
be implausible for a topological sort on a DAG): user time of 9s.

I conclude that git chose the right representation for a DAG of
changesets right off the bat.  It is <effect sound="drum roll" />
a DAG of changesets.

AFAICT, both bazaar and Mercurial said, "whoa, that's going to be
inefficient in some operations", and immediately chose array
representations with some kind of temporal index (temporal because
they want all inserts in the database to be file appends for ACID and
efficiency).  Linus said "Listen to Tony", and he proved right when
the "git pack" format was introduced.

Original Arch and Darcs are "set of changeset"-oriented, so they can't
be compared to the DAG-oriented ones on this dimension; they can do
things directly that git, bzr, and hg can't (although I wouldn't put
it past Linus and/or other gitfans to implement efficient indirect
ways to accomplish them).

This is a consequence of what I posted a couple of months ago.  git is
a Lisp specialized to manipulating revision DAGs and accessing the
underlying revisions.

None of this is intended to specifically advocate git for the Emacs
repo.  Mercurial is probably good enough (an estimate after using it
with XEmacs, a comparable codebase but only about 500 revisions, since
early December).  bzr is likely good enough too.  Both have UI
advantages over git at the moment, and bzr log is definitely the best
(if you ignore how slow it is), but why bother with "bzr log" when
"gitk" is almost certainly equally fast (and much faster to update
than bzr log is to run again)?

OTOH, many Emacs developers won't run any vcs by hand, they'll use
vc.el or pcl-<vcs>.el (which doesn't exist for vcs in (git hg bzr)
AFAIK, but if Emacs chooses one, I bet it's a matter of days).  In
that case it's important that *all* vc and pcl operations be
acceptably efficient.

Re: Emacs Bazaar repository

[Thomas Lord]

Stephen J. Turnbull wrote:
> Question in my mind is, given bzr's propensity to change
> representations, does the bzr development team have somebody who's
> worked on that kind of function, and demands that functionality be
> rock-solid?
>
>   

"Back in the day" (early days of GNU), aside from walking
uphill to school in both directions, the project had a pretty
solid revision control history *across the various programs*.

Namely, the GNU FTP site.   It had (approximately) every
officially named revision of all GNU programs.   It had (approximately)
all of the "interesting" diffs between versions.   We did "DVC"
by hand (and we liked it that way!).    The concept of a
CVS "vendor branch" comes right out of this arrangement.

It's been downhill ever since.   Arch had the right idea which
is to just automate that old situation rather than replace it.
For performance, build *ancillary* indexes and databases
but treat the tar bundles as the end-to-end check;  those
are things that are ultimately being managed.

-t



>
>   

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Incidentally, why are we concentrating on "bzr log"?

Partly because intuitively it "should" be fast, it's just a "cat" of
the index file.  But this isn't true for bzr, so people's expectations
are contradicted.

 > is that a frequent operation?

Yes.  You need changeset IDs frequently for communicating with users
and other developers, and for diffs.

 > With CVS, I find myself doing "cvs log" only once in a few months,
 > when I'm looking for a change corresponding to some ChangeLog
 > entry.

I think that you will pretty quickly find that ChangeLogs as you know
them have to be abandoned, because they cause merge conflicts at
O(n^2) or so, where n = "rate of commits/unit of time".  Also, most
developers in projects using VCSes more recent than CVS use "<vcs>
log" a lot.  A big problem is that ChangeLogs either have to be
excluded from their changesets, or you can't include a changeset ID in
the ChangeLog entry corresponding to the changeset.  (IDs are secure
hashes and thus you must solve a fixed-point problem for a secure hash
function if the ChangeLog containing the ID is part of the changeset!)

 > Aren't "push" and "pull" much more important, as far as speed is
 > concerned, for everyday work?

In theory, yes.  In practice, no.  They're done sufficiently often
that (with the possible exception of Darcs) all recent VCSes are
net-lag-bound because users won't put up with speed issues for local
repos.  In practice these also only have "big O" problems when you're
pulling an old and active branch for the first time, which is
acceptable if success is guaranteed.  (Another problem with bzr, as
we've seen -- a long pull is likely to fail because the requested
archives may disappear before they get shipped out the wire.  This is
probably a configuration issue with Jason's repo, typically there are
parameters you want to tweak for public repos vs private branches.)

 > Also the equivalent of "cvs diff", I think.

With the exception of Darcs (which can be O(r1 - r2)), all modern
VCSes do this quickly.

 > Those are the ops I use much more frequently than "log" and "annotate".

I use both fairly frequently, and it's very annoying to me when they
take more than 5 seconds or so.  YMMV of course.

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

James Westby writes:

 > May I point you to my blog post of yesterday that explains some of
 > these issues, rather than spell if all out here?
 > 
 > http://jameswestby.net/weblog/bzr/01-revision-numbers.html
 > 
 > The last section is on this particular issue.
 > 
 > The gist of it is that topological sorting means that children appear
 > before their parents. bzr does "merge sorting" that ensures that a
 > revision does not appear after a revision that it is not an ancestor
 > of.

Excuse me?  There may be something sensible you wish to say, but what
I have quoted is not it.  If A and B are not ordered by ancestry both
"AB" and "BA" fail to be a "merge sort" according to that criterion.

From the blog:

  Always having merge commits means that "bzr log --short" and "bzr
  log --line" can give you a good summary of what happened on your
  branch, the commits you did, and the things that you merged.  It
  preserves a mainline for you in the left hand ancestry,

git-show-branch does this satisfactorily for me, although admittedly
more recent unmerged commits on other branches may show up first.
This is a *good* thing if I'm thinking about merging them, though.
OTOH, gitk simply prunes any unmerged commits by default.  How does
"merge sort" differ from "topological sort pruning unmerged commits"?
AFAICT in that case "not an ancestor" == "is a descendant" so topo
sort and merge sort are the same.  What am I missing?

  The indentation of the merged commits (and the fact they disappear
  with "--short" and "--line") means that mentally they become of
  lesser importance.

But that's exactly what I rarely want.  I know what I did, unless it
was a long time ago.  What I generally want to know is what others are
doing, and how that impacts my branch when I merge their code.

 > While we may be able to offer some UI improvements to git itself there
 > may be fundamental differences that mean git may always fall short of
 > what we would like. There are also things that bzr can do that git
 > will never do (barring any big changes in direction), for instance
 > foreign branch support.

What is "foreign branch support", and why do you think that a program
which is amounts to being a browser for the universal revision graph
can't do it?  An URL would do.

Re: Emacs Bazaar repository

[Jonathan Rockway]

* On Fri, Mar 14 2008, Lennart Borgman (gmail) wrote:
>>  If Mercurial had the ability to truly support multiple branches in
>> the same folder (with out requiring me to merge all branches before I
>> can pull - pull works only if there is a single tip/branch), I would
>> have preferred it mainly because it just needs PYTHON and nothing else
>> (GIT needs PERL and SHELL).
>
>
> How did they do that? It needs both perl and sh? Is there really any
> perl programmer who writes code that way?

I don't think they mix Perl and shell in the same files.  It's just that
git is a collection of written-in-C "plumbing" and then some helpful
scripts on top of that plumbing.  Some of the helpful scripts are
written in Perl, some are written in shell.

So to use git, you need a C compiler, a UNIX shell, and Perl.  But I
think that setup is actually more common than Python, so it's not a real
loss.  (Except on Windows that doesn't have a shell or a as-good-as-UNIX
perl.  But Windows is lacking in a lot of other features too.)

Finally, for the record, git is not a good place to learn Perl from :)
But hey, it works.

Regards,
Jonathan Rockway

-- 
print just => another => perl => hacker => if $,=$"

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

Lennart Borgman (gmail) writes:

 > How did they do that? It needs both perl and sh? Is there really any 
 > perl programmer who writes code that way?

"They" is the operative word.  Some code was written in C by C
programmers, some as shell scripts by shell programmers, some in Perl
by Perl programmers, some in Tcl by Tcl programmers, and some even in
Python IIRC.

Re: Emacs Bazaar repository

[Harald Meland]

[Andreas Schwab]

> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>
>> Andreas Schwab <schwab@suse.de> writes:
>>
>>> Yes, "diff" is quite important as well.  Unfortunately, bzr diff seems
>>> to have the same performance problem.
>>
>> It shouldn't be on a "normal" setup at least with a recent enough bzr.
>> Git is obviously faster, but for example, on an emacs tree, "bzr diff"
>> takes 0.3 seconds, while Git takes 0.05. So, the time taken by bzr is
>> acceptable to me.
>
> What is the fastest way to get the difference of a revision relative to
> its ancestor?

I would guess "bzr diff -r before:revid:$revision_id..revid:$revision_id"
(which ought to be equivalent to "bzr diff -c revid:$revision_id") as
that (in theory) shouldn't need to calculate any revision numbers.

Re: Emacs Bazaar repository

[Stephen J. Turnbull]

Martin Pool writes:

 > This means that c and d were merged into this branch in revision b,
 > and were not previously present in the repository.  This layout gives
 > a good overview of the history, in our opinion a much better one than
 > what you get from tools that just print one path through the graph or
 > that print the revisions in arbitrary linear order.

OK, but is it worth a minute per execution when a minute will give me
a gitk view of the history?

 > To calculate which revisions are newly merged is currently done by
 > examining the whole graph.  We will do two things to speed it up:
 > faster loading of the graph from the repository, and caching some data
 > in the branch.

Well, as I pointed out in another post, in git the repo *is* the DAG.
No caches needed.

 > But possibly you don't want to know about merged in revisions, and
 > that can be done with bzr log --short or --line.  These should be able
 > to do much less work, just walking down through the revision to print.

I always want to know about the merged-in revisions.  It would never
occur to me to leave them out ... unless that was the only way to get
a sub-minute log, of course.  I'm really having trouble grasping what
kind of workflow this tool is intended for.

 > I want to point out that bzr branch locally, not in a single
 > repository, is doing more than just copying all the files - it is
 > filtering out and verifying the data relevant to the particular branch
 > you're copying.

But so does git fetch.  Of course, since the repo is the DAG, it
doesn't have to do any extra work.  If it finds the object, the DAG is
correct.  If it doesn't, the repo is corrupt.

I don't know the details of the git packed format, but since indicies
and content are in separate files, presumably the content packs could
be corrupt even though the DAG checks out.  But that will be caught
O(0) because git does a checksum for every transmission anyway.  I
assume that's true for bzr as well, of course.

 > >  # Command line typo on a local URL, bzr figures it out in 10s.
 > >  $ time bzr branch EMACS_22_BASE
 > >  bzr: ERROR: Not a branch: "/Users/steve/Software/Emacs/emacs-bzr/EMACS_22_BASE/".
 > >
 > >  real    0m10.183s       user    0m0.457s        sys     0m1.361s
 > 
 > Maybe you could send (perhaps just on the bzr list) a run of that with
 > "bzr --lsprof branch ...".  It may be that the low cpu usage is caused
 > by many things being pushed out of ram by building the emacs working
 > tree in the previous command.

That's really cold comfort.  My CPU and I/O bus are typically
multitasked.  Anyway, "user 0.457s + sys 1.361s" is over 10 times my
threshold of perception.  Presumably that's essentally all Python
startup overhead, since there was no branch to look at there.  That's
a real handicap against git, where "git-rev-list --topo" posts "user
0.060s + sys 0.124s" for >600 revs!

I ran "time bzr --lsprof branch EMACS_22_BASE" twice in succession.
The first was just over 10s wallclock again, the second down to just
under 6s.  URLs to profiles follow time output.

real    0m10.097s	user    0m1.109s	sys     0m1.828s
real    0m5.954s	user    0m0.768s	sys     0m1.201s

http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/bzf.prof.0
http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/bzf.prof.1

Re: Emacs Bazaar repository

[James Westby]

On Sat, 2008-03-15 at 09:39 +0900, Stephen J. Turnbull wrote:
> James Westby writes:
> 
>  > May I point you to my blog post of yesterday that explains some of
>  > these issues, rather than spell if all out here?
>  > 
>  > http://jameswestby.net/weblog/bzr/01-revision-numbers.html
>  > 
>  > The last section is on this particular issue.
>  > 
>  > The gist of it is that topological sorting means that children appear
>  > before their parents. bzr does "merge sorting" that ensures that a
>  > revision does not appear after a revision that it is not an ancestor
>  > of.
> 
> Excuse me?  There may be something sensible you wish to say, but what
> I have quoted is not it.  If A and B are not ordered by ancestry both
> "AB" and "BA" fail to be a "merge sort" according to that criterion.
> 

Sorry, it appears I didn't explain it well enough. I missed the word
"also", as bzr also adds a constraint over and above topo-sorting.
It then also prefers to output left-hand parents first.

Consider the following revision graph (parents above children)

         A
         |\
         B C
         |/
         D

A topo-sort requires that D be emitted before B and C, and that B and C
be emitted before A, but it does not impose an ordering on B and C.

If you add bzr's rules then it prefers to output left-hand parents
first, so it would output B before C. However this would then violate
the other constraint, as C would come after B, but B is not an ancestor
of C. Therefore C must be output before B. Add in the indentation
depending on whether the revision is a left or right hand parent
of the previous revision and you have bzr's log output.

>   The indentation of the merged commits (and the fact they disappear
>   with "--short" and "--line") means that mentally they become of
>   lesser importance.
> 
> But that's exactly what I rarely want.  I know what I did, unless it
> was a long time ago.  What I generally want to know is what others are
> doing, and how that impacts my branch when I merge their code.
> 

Ok, use "bzr merge --pull".

>  > While we may be able to offer some UI improvements to git itself there
>  > may be fundamental differences that mean git may always fall short of
>  > what we would like. There are also things that bzr can do that git
>  > will never do (barring any big changes in direction), for instance
>  > foreign branch support.
> 
> What is "foreign branch support", and why do you think that a program
> which is amounts to being a browser for the universal revision graph
> can't do it?  An URL would do.
> 
> 

I'm sorry, I can't really follow this paragraph. I can explain what we
mean by foreign branch support.

We mean that the bzr client can access the branches/repositories/working
trees of other systems transparently. For instance if you install 
bzr-svn, "bzr branch svn://..." works.

This might not have been the best example, as git-svn exists, and works
great for using git to access svn. However it is not transparent, you
must use the git-svn command for some tasks. Also you can't use the
git client in an svn working tree (checkout) if you have one already.

Thanks,

James

Re: Emacs Bazaar repository

[dhruva]

Hi,

On Sat, Mar 15, 2008 at 3:56 AM, Martin Geisler <mg@daimi.au.dk> wrote:
>  And even if you have several heads in your Mercurial repository, you can
>  certainly still pull in new changesets.

I have seen it and have used it too. The problem comes (from my
experience) when you have to push. I agree you can create a new named
branch and just pull in changes or do a forced pull which will create
a new head. Suppose I want to keep multiple named branches active and
yet push to a remote repository from one of the named branch, IMO it
is not possible with mercurial. I would be happy to know if there is a
way to do it.

-dhruva

Re: Emacs Bazaar repository

[James Westby]

On Sat, 2008-03-15 at 10:30 +0000, James Westby wrote:
> Sorry, it appears I didn't explain it well enough. I missed the word
> "also", as bzr also adds a constraint over and above topo-sorting.
> It then also prefers to output left-hand parents first.

Yeah, these rules don't actually describe the algorithm at all,
sorry about that.

How about "bzr outputs all descendants of the right hand parents
of a revision before the left hand parent, except those that are
also ancestors of the left hand parent?" Does that describe the
algorithm?

Thanks,

James
 

Re: Emacs Bazaar repository

[Eli Zaretskii]

> From: Jonathan Rockway <jon@jrock.us>
> Date: Fri, 14 Mar 2008 19:43:05 -0500
> 
> Windows is lacking in a lot of other features too.

Actually, it doesn't, it just does it differently.  For example, the
shell in the latest versions is quite powerful, but of course is not
compatible with Posix shells.

Re: Emacs Bazaar repository

[dhruva]

Hi,

On Sat, Mar 15, 2008 at 8:07 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Jonathan Rockway <jon@jrock.us>
>  > Date: Fri, 14 Mar 2008 19:43:05 -0500
>  >
>  > Windows is lacking in a lot of other features too.
>
>  Actually, it doesn't, it just does it differently.  For example, the
>  shell in the latest versions is quite powerful, but of course is not
>  compatible with Posix shells.

I resisted from responding. Having worked on UNIX (and various
flavors), M$ and OpenVMS too, I must say each operating system has a
different way of doing things. Just because they are not POSIX
compliant does not mean it is bad or unusable. The M$ bashing is a bit
old fashioned these days and it makes sense to develop applications
that can be used on multiple platforms as the user base is quite
evenly distributed.

-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Martin Geisler]

[-- Attachment #1.1: Type: text/plain, Size: 2043 bytes --]

dhruva <dhruvakm@gmail.com> writes:

> Hi,
>
> On Sat, Mar 15, 2008 at 3:56 AM, Martin Geisler <mg@daimi.au.dk> wrote:
>>dhruva <dhruvakm@gmail.com> writes:
>>> Having tried a bunch of SCM, I must say GIT way of supporting
>>> multiple branches under the same folder along with its speed it a
>>> sure winner. I was opposing GIT due to its non-availability on M$,
>>> it is history now and the port they have is really good. The build
>>> is streamlined on M$ too, I pull their changes regularly, build,
>>> install and use the bleeding edge. It has not failed me so far.
>>>
>>> If Mercurial had the ability to truly support multiple branches in
>>> the same folder (with out requiring me to merge all branches before
>>> I can pull - pull works only if there is a single tip/branch), I
>>> would have preferred it mainly because it just needs PYTHON and
>>> nothing else (GIT needs PERL and SHELL).
>>
>> I think you are confusing several 'heads' with several 'branches'
>> here. And even if you have several heads in your Mercurial
>> repository, you can certainly still pull in new changesets.
>
> I have seen it and have used it too. The problem comes (from my
> experience) when you have to push. I agree you can create a new named
> branch and just pull in changes or do a forced pull which will create
> a new head. Suppose I want to keep multiple named branches active and
> yet push to a remote repository from one of the named branch, IMO it
> is not possible with mercurial. I would be happy to know if there is a
> way to do it.

I'm afraid I'm no expert on named branches in Mercurial, I just wanted
to point out that pulling in changesets is a normal supported operation
even when you have multiple heads.

I've posted this to the Mercurial user list too since I think the
question is much more suited for them to answer.

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multi-Party Computation) to Python. See: http://viff.dk/.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Emacs Bazaar repository

[Stefan Monnier]

> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:

Another piece of information that I find missing is all the tags.
It seems they are present in the form of
http://bzr.notengoamigos.org/emacs/tags pseudo branches, but they're not
present in the form of actual Bzr tags.  Also some important ones (such
as EMACS_19_34, EMACS_20_4, and EMACS_21_1) are missing.


        Stefan

Re: Emacs Bazaar repository

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Also some important ones (such as EMACS_19_34, EMACS_20_4, and
> EMACS_21_1) are missing.

These are due to shortcomings of cvsps.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Toshio Kuratomi]

[-- Attachment #1.1: Type: text/plain, Size: 716 bytes --]

Stefan Monnier wrote:
>> After a few false starts I have been successful (at least as far as I
>> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
>> is interested in playing with the results you can check out the HEAD of
>> the CVS repository with:
> 
> Another piece of information that I find missing is all the tags.
> It seems they are present in the form of
> http://bzr.notengoamigos.org/emacs/tags pseudo branches, but they're not
> present in the form of actual Bzr tags.  

I wrote a patch about six months to a year ago to deal with this.
https://bugs.launchpad.net/bzr-cvsps-import/+bug/128326

I'm attaching the patch -- but it might need to be updated.

-Toshio

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: bzr-cvsps-tags.patch --]
[-- Type: text/x-patch; name="bzr-cvsps-tags.patch", Size: 8580 bytes --]

=== modified file '__init__.py'
--- __init__.py	2007-01-19 17:03:04 +0000
+++ __init__.py	2007-07-25 16:36:34 +0000
@@ -64,10 +64,14 @@
                                    help='Use cvs to extract texts.'),
                      option.Option('use-rcs',
                                    help='Use rcs to extract texts. (default)'),
+                     option.Option('use-dirstate-tags',
+                                   help='Use bzr metadata tags rather than'
+                                   ' a branch per tag'),
                     ]
 
     def run(self, cvsroot=None, module=None, output=None, cvsps_dump=None,
-            encoding=None, verify=True, use_cvs=False, use_rcs=False):
+            encoding=None, verify=True, use_cvs=False, use_rcs=False,
+            use_dirstate_tags=False):
         from cvsps import importer
 
         if cvsroot.startswith(':pserver:') or cvsroot.startswith(':ext:'):
@@ -78,11 +82,18 @@
         if not use_cvs and not use_rcs:
             # Default is to use rcs, since it is slightly faster.
             use_cvs = False
+
+        if use_dirstate_tags:
+            tag_style = 'dirstate'
+        else:
+            tag_style = 'branch'
+
         importer = importer.Importer(cvsroot, module, output,
                                      cvsps_dump=cvsps_dump,
                                      encoding=encoding,
                                      verify=verify,
-                                     use_cvs_for_text=use_cvs)
+                                     use_cvs_for_text=use_cvs,
+                                     tag_style=tag_style)
         importer.process()
 
 

=== modified file 'cvsps/importer.py'
--- cvsps/importer.py	2007-02-12 18:03:03 +0000
+++ cvsps/importer.py	2007-07-25 18:14:22 +0000
@@ -597,11 +597,12 @@
     """
 
     def __init__(self, bzr_repo, cvs_root, cvs_module, map_file,
-                 verify=True, use_cvs_for_text=True):
+                 verify=True, use_cvs_for_text=True, tag_style=None):
         self._bzr_repo = bzr_repo
         self._map_file = map_file
         self._cvs_root = cvs_root
         self._cvs_module = cvs_module
+        self._tag_style = tag_style or 'branch'
 
         self._working_path = osutils.pathjoin(
             self._bzr_repo.bzrdir.root_transport.local_abspath('.'),
@@ -623,7 +624,7 @@
         self._n_existing_patches = 0
         self._n_tags = 0
 
-    def handle_patchset(self, patchset):
+    def handle_patchset(self, patchset, pb):
         """Handle one of the patchsets from cvs to bzr"""
 
         revision_id = self._map_file.get(patchset.num)
@@ -653,7 +654,11 @@
             self._n_patches += 1
 
         if patchset.tag is not None:
-            self._handle_tag(patchset, revision_id)
+            if self._tag_style == 'dirstate':
+                self._handle_tag_dirstate(patchset, revision_id, pb)
+            else:
+                self._handle_tag_branch(patchset, revision_id)
+
             action += '+tag'
 
         return revision_id, action
@@ -830,8 +835,13 @@
         os.makedirs(branch_path)
 
         # Create a new one
+        format = bzrdir.BzrDirFormat.get_default_format()
+        if self._tag_style == 'dirstate' and not format.get_branch_format().supports_tags():
+            format = bzrdir.format_registry.get('dirstate-tags')()
+
         target_branch = bzrdir.BzrDir.create_branch_convenience(branch_path,
-                                                         force_new_tree=False)
+                                                         force_new_tree=False,
+                                                         format=format)
         self._set_repo(target_branch)
         target_branch.lock_write()
         self._cache_branch(patchset.branch, target_branch)
@@ -875,7 +885,23 @@
 
         return revision_id
 
-    def _handle_tag(self, patchset, revision_id):
+    def _handle_tag_dirstate(self, patchset, revision_id, pb):
+        """Create a tag with the given revision id."""
+        # TODO: toshio 20070725 We probably want to check that the branch
+        # format supports tags and convert it if it doesn't.  However, the
+        # repository is locked at this point so it's not as simple as the
+        # below code.
+        #
+        # Currently, I'm making sure that the branches support tags when
+        # we create them.
+        #if not self._cur_bzr_branch.supports_tags():
+        #    newFormat = bzrdir.format_registry.get('dirstate-tags')()
+        #    converter = self._cur_bzr_branch.bzrdir._format.get_converter(newFormat)
+        #    self._cur_bzr_branch = converter.convert(self._cur_bzr_branch.bzrdir, pb)
+        self._cur_bzr_branch.tags.set_tag(patchset.tag, revision_id)
+        self._n_tags += 1
+
+    def _handle_tag_branch(self, patchset, revision_id):
         """Create a tag with the given revision id."""
         tag_branch_path = self._get_tag_branch_path(patchset.tag)
         try:
@@ -969,10 +995,12 @@
     """Import a CVS project into bzr."""
 
     def __init__(self, cvsroot, cvs_module, output_base, cvsps_dump=None,
-                 encoding=None, verify=True, use_cvs_for_text=True):
+                 encoding=None, verify=True, use_cvs_for_text=True,
+                 tag_style=None):
         self._cvs_root = osutils.abspath(cvsroot)
         self._cvs_module = cvs_module
         self._use_cvs_for_text = use_cvs_for_text
+        self._tag_style = tag_style or 'branch'
         self._verify = verify
 
         self.output_base = output_base
@@ -1016,7 +1044,7 @@
                 os.makedirs(path)
         self._paths_created = True
 
-    def open_or_create_bzr_repo(self):
+    def open_or_create_bzr_repo(self, pb):
         """Open the bzr repository, creating it if needed."""
         self.setup_directories()
         bzr_repo_transport = transport.get_transport(self._repo_path)
@@ -1024,6 +1052,11 @@
             a_bzrdir = bzrdir.BzrDir.open_from_transport(bzr_repo_transport)
         except errors.NotBranchError:
             return self._create_bzr_repo(bzr_repo_transport)
+        if self._tag_style == 'dirstate' and (
+                not a_bzrdir.find_branch_format().supports_tags()):
+            newFormat = bzrdir.format_registry.get('dirstate-tags')()
+            converter = a_bzrdir._format.get_converter(newFormat)
+            a_bzrdir = converter.convert(a_bzrdir, pb)
         return a_bzrdir.open_repository()
 
     def _create_bzr_repo(self, a_transport):
@@ -1033,6 +1066,10 @@
         except errors.FileExists:
             pass
         fmt = bzrdir.BzrDirFormat.get_default_format()
+        if self._tag_style == 'dirstate' and not (
+                fmt.get_branch_format().supports_tags()):
+            fmt = bzrdir.format_registry.get('dirstate-tags')()
+
         control = fmt.initialize_on_transport(a_transport)
         repo = control.create_repository(shared=True)
         repo.set_make_working_trees(False)
@@ -1065,7 +1102,7 @@
         n_patchsets = len(patchsets)
         for i, patchset in enumerate(patchsets):
             try:
-                rev_id, action = cvs_to_bzr.handle_patchset(patchset)
+                rev_id, action = cvs_to_bzr.handle_patchset(patchset, pb)
             except KeyboardInterrupt:
                 if pb is not None:
                     pb.clear()
@@ -1101,7 +1138,11 @@
 
     def process(self):
         """Start converting the repository."""
-        repo = self.open_or_create_bzr_repo()
+        pb = ui.ui_factory.nested_progress_bar()
+        try:
+            repo = self.open_or_create_bzr_repo(pb=pb)
+        finally:
+            pb.finished()
         # Maintain a repository wide lock for the whole transaction
         # that should help cache stuff.
         # TODO: jam 20061121 This may actually cache *too* much. Consider
@@ -1115,7 +1156,8 @@
             try:
                 cvs_to_bzr = CVSToBzr(repo, self._cvs_root, self._cvs_module,
                                       map_file, verify=self._verify,
-                                      use_cvs_for_text=self._use_cvs_for_text)
+                                      use_cvs_for_text=self._use_cvs_for_text,
+                                      tag_style=self._tag_style)
                 start_time = time.time()
                 pb = ui.ui_factory.nested_progress_bar()
                 try:


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Matthieu Moy]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> OTOH, many Emacs developers won't run any vcs by hand, they'll use
> vc.el or pcl-<vcs>.el (which doesn't exist for vcs in (git hg bzr)
> AFAIK, but if Emacs chooses one, I bet it's a matter of days).

It's not called pcl-<vcs>, but DVC exists:

http://download.gna.org/dvc/

(don't look at the manual, prehistorically deprecated, but the tool
itself is quite good, and supports Bzr, Baz, Mercurial, Monotone,
Darcs and Git).

Re: Emacs Bazaar repository

[Matthieu Moy]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Lennart Borgman (gmail) writes:
>
>  > How did they do that? It needs both perl and sh? Is there really any 
>  > perl programmer who writes code that way?
>
> "They" is the operative word.  Some code was written in C by C
> programmers, some as shell scripts by shell programmers, some in Perl
> by Perl programmers, some in Tcl by Tcl programmers, and some even in
> Python IIRC.

There used to be a few lines of python in Git itself I think, but it
has been rewritten to remove the python dependency. The hg and p4
importers are in contrib/, and written in python though.

Re: Emacs Bazaar repository

[Sergei Organov]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
[...]
> OTOH, many Emacs developers won't run any vcs by hand, they'll use
> vc.el or pcl-<vcs>.el (which doesn't exist for vcs in (git hg bzr)
> AFAIK, but if Emacs chooses one, I bet it's a matter of days).

There is git.el in the GIT source tree that is intended to mimic pcl-cvs
as much as possible and is already quite useful. There is also
git-annotate.el.

-- Sergei.

Re: Emacs Bazaar repository

[Michael Haggerty]

Andreas Schwab wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
>> Also some important ones (such as EMACS_19_34, EMACS_20_4, and
>> EMACS_21_1) are missing.
> 
> These are due to shortcomings of cvsps.

Please be very careful with conversion tools that are based on cvsps.
It is an admirable attempt at the very difficult job of allowing
incremental operation, but it is known to give significantly incorrect
output in many circumstances.  See for example [1,2,3].  Note that
git-cvsimport is based on a version of cvsps that they improved to fix
some problems, but this version can also output history that is
completely incorrect.

Please consider using cvs2svn for your conversion [4].  It can output to
git-fast-import (and therefore hopefully to bzr-fast-import) [5], it
avoids a number of the known limitations of cvsps which can result in
incorrect history, it can easily handle the emacs CVS repository (at
least when converting to SVN), and it has very many options that can be
used to customize the conversion [6].  It has a long track record of
converting from CVS to SVN (though it can only be used for one-time
conversions).

Having said that, using cvs2svn to output to a DVCS is rather new and
has a couple of known problems.  One is that too many files are added to
branches and tags (because I misunderstood an aspect of the
git-fast-import format); the other is that it creates fixup branches for
all tags even when they are not needed.  I am currently working on both
of these problems, but you might want to wait a bit before using it for
a final conversion.

Whatever tool you use to do the conversion, please check at least the
contents of the latest trunk, tags, and branches before relying on the
conversion output.  I'd hate to see the venerable emacs history
corrupted :-)

Michael

[1] http://marc.info/?l=git&m=118260312708709&w=2

[2] http://common-lisp.net/pipermail/slime-devel/2008-March/007173.html

[3] http://selenic.com/pipermail/mercurial-devel/2008-February/004975.html

[4] http://cvs2svn.tigris.org

[5] http://cvs2svn.tigris.org/cvs2git.html

[6] http://cvs2svn.tigris.org/features.html

Re: Emacs Bazaar repository

[Andreas Schwab]

Michael Haggerty <mhagger@alum.mit.edu> writes:

> Andreas Schwab wrote:
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> 
>>> Also some important ones (such as EMACS_19_34, EMACS_20_4, and
>>> EMACS_21_1) are missing.
>> 
>> These are due to shortcomings of cvsps.
>
> Please be very careful with conversion tools that are based on cvsps.
> It is an admirable attempt at the very difficult job of allowing
> incremental operation, but it is known to give significantly incorrect
> output in many circumstances.

I know, I'm currently trying to fixup some of the mistakes it made.

> Please consider using cvs2svn for your conversion [4].

While I agree that cvs2svn is the best tool available to convert a CVS
repository, the biggest problem with it is that it is not incremental.
Having to start from scratch would throw away all the nice fixup made to
create proper merges (and there are _many_ merges in Emacs CVS).

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Dan Nicolaescu]

Sergei Organov <osv@javad.com> writes:

  > "Stephen J. Turnbull" <stephen@xemacs.org> writes:
  > [...]
  > > OTOH, many Emacs developers won't run any vcs by hand, they'll use
  > > vc.el or pcl-<vcs>.el (which doesn't exist for vcs in (git hg bzr)
  > > AFAIK, but if Emacs chooses one, I bet it's a matter of days).
  > 
  > There is git.el in the GIT source tree that is intended to mimic pcl-cvs
  > as much as possible and is already quite useful. 

The plan is that vc-status should at some point supersede pcl-cvs and
git.el.  It can be used now, but it's not complete.

  > There is also git-annotate.el.

This should not be needed anymore, vc-git.el supports vc-annotate.

Re: Emacs Bazaar repository

[dhruva]

Hello,
 With lengthy discussions having taken place on this SCM issue, what
are we waiting for? Are there really any concrete plans to move to a
dSCM or are we just wasting talking about features and shortcomings of
each tool. Any outcome of ESR study?

 From my truly humble opinion, GIT and Mercurial are the top
contenders with GIT having an edge over Mercurial due to speed,
ability to hack on multiple branches in the same folder (repository)
and an active development and porting initiatives (mercurial has
similar record). GIT has the ability to talk to a SVN repository,
though this is not important for Emacs, more people might start using
GIT because of this feature (well, I started mainly due to this
feature as my work place used SVN). The fact that GIT has a CVS server
option makes it possible to share the code base to CVS clients. For
example, if someone on OpenVMS (DEC VMS) wants to access Emacs
repository, GIT is not ported to VMS. They can still use the port of
CVS client on VMS to access Emacs source directly from VMS.

-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Karl Fogel]

dhruva <dhruvakm@gmail.com> writes:
>  With lengthy discussions having taken place on this SCM issue, what
> are we waiting for? Are there really any concrete plans to move to a
> dSCM or are we just wasting talking about features and shortcomings of
> each tool. Any outcome of ESR study?

I think you may have missed the news :-).

We decided on bzr, which in practice seems to mean "some people are
seriously testing bzr, and once we have a good conversion and are
satisfied we can get work done, someone will install it on savannah and
that repository will be the new master".

All of the DVCSs seem good.  No one marshalled any compelling arguments
in favor of one versus the other on technical grounds, and all other
things being equal, RMS (and maybe some others, perhaps including Yidong
and Stefan?) preferred bzr because it is a GNU project.

The above is a summary of what I take to be the current state of things.
My personal opinions: I'm also happy with bzr, and would be equally
happy with any of the usual suspects among free software DVCS.  I just
hope we can switch the master soon.

-Karl

Re: Emacs Bazaar repository

[dhruva]

Good that the decision has been made. Since it is bzr, the earlier we
have an official bzr emacs repo the better it is. I can focus my
efforts on testing bzr and hope to see its performance improved. Like
the git repo on savannah, we need a bzr repo to play with real fast.

-dhruva

On 3/18/08, Karl Fogel <kfogel@red-bean.com> wrote:
> dhruva <dhruvakm@gmail.com> writes:
> >  With lengthy discussions having taken place on this SCM issue, what
> > are we waiting for? Are there really any concrete plans to move to a
> > dSCM or are we just wasting talking about features and shortcomings of
> > each tool. Any outcome of ESR study?
>
> I think you may have missed the news :-).
>
> We decided on bzr, which in practice seems to mean "some people are
> seriously testing bzr, and once we have a good conversion and are
> satisfied we can get work done, someone will install it on savannah and
> that repository will be the new master".
>
> All of the DVCSs seem good.  No one marshalled any compelling arguments
> in favor of one versus the other on technical grounds, and all other
> things being equal, RMS (and maybe some others, perhaps including Yidong
> and Stefan?) preferred bzr because it is a GNU project.
>
> The above is a summary of what I take to be the current state of things.
> My personal opinions: I'm also happy with bzr, and would be equally
> happy with any of the usual suspects among free software DVCS.  I just
> hope we can switch the master soon.
>
> -Karl
>

-- 
Sent from Gmail for mobile | mobile.google.com

Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Jonathan Lange]

On Tue, Mar 18, 2008 at 4:00 PM, dhruva <dhruvakm@gmail.com> wrote:
> Good that the decision has been made. Since it is bzr, the earlier we
>  have an official bzr emacs repo the better it is. I can focus my
>  efforts on testing bzr and hope to see its performance improved.

It's improving even as we talk about it. There have already been
performance patches filed because of this thread[1].

It'd be great if you keep raising performance issues on the Bazaar
mailing list or on IRC (#bzr on Freenode) -- it'll make them a lot
easier to fix.

jml


[1] This one in particular,
http://bundlebuggy.aaronbentley.com/request/%3C47DB7DC7.8080400@arbash-meinel.com%3E
and one mentioned in a previous post
http://bundlebuggy.aaronbentley.com/request/%3C47D9E60A.6030203@canonical.com%3E

Re: Emacs Bazaar repository

[Andreas Schwab]

Karl Fogel <kfogel@red-bean.com> writes:

> All of the DVCSs seem good.  No one marshalled any compelling arguments
> in favor of one versus the other on technical grounds, and all other
> things being equal, RMS (and maybe some others, perhaps including Yidong
> and Stefan?) preferred bzr because it is a GNU project.

In its current form bzr is not usable for a project of the size of
Emacs.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[dhruva]

Hello,

On Tue, Mar 18, 2008 at 2:47 PM, Andreas Schwab <schwab@suse.de> wrote:
> Karl Fogel <kfogel@red-bean.com> writes:
>
>  > All of the DVCSs seem good.  No one marshalled any compelling arguments
>  > in favor of one versus the other on technical grounds, and all other
>  > things being equal, RMS (and maybe some others, perhaps including Yidong
>  > and Stefan?) preferred bzr because it is a GNU project.
>
>  In its current form bzr is not usable for a project of the size of
>  Emacs.

I feel the decision to move to bzr is mainly because it is a GNU
project. Every open discussion seems to suggest either GIT or
mercurial in the emacs list (and other projects considering a change
in SCM too).
Either we decide to postpone till bzr is good enough or set a timeline
by which all SCM related comparisons can happen and decide. Deciding
to move the bzr and then hope for things to improve will be depending
too much on HOPE.
 I just hope it the decision is based on technical facts rather than
affiliations and emotions...

-dhruva

-- 
Contents reflect my personal views only!

Emacs repository benchmark: bzr and git

[Teemu Likonen]

I did some benchmarking in git and bzr repositories of Emacs. Some
numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"), 2825
files. Both repositories seem to have just linear history converted from
CVS repo. Both have the same head revision which is
481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).

Repositories/branches are pulled from here:
git: git://git.sv.gnu.org/emacs.git
bzr: http://bzr.notengoamigos.org/emacs/trunk/

My system is AMD Sempron 3000+ with 2 GB memory and it's running Debian
GNU/Linux 4.0. I'm using the latest development versions of both git
(1.5.5.rc0.6.gdeda) and bzr (1.4dev). I just measured with 'time'
command how long it takes to run certain commands.



Viewing history
---------------


The complete history:

$ time git log >/dev/null
real    0m5.741s

$ time bzr log >/dev/null
real    3m15.708s


Last 100 revisions:

$ time git log -100 >/dev/null
real    0m0.011s

$ time bzr log -l100 >/dev/null
real    2m10.270s


Last 10 revisions:

$ time git log -10 >/dev/null
real    0m0.007s

$ time bzr log -l10 >/dev/null
real    2m9.163s


The complete history of a single file:

$ time git log src/keymap.c >/dev/null
real    0m9.240s
(The same as above but with detecting and following possible renames:)
$ time git log --follow src/keymap.c >/dev/null
real    0m17.891s

$ time bzr log src/keymap.c >/dev/null
real    3m35.431s



Differences between revisions
-----------------------------


I'm using exactly the same revisions in both repositories:

revid:cvs-1:wohler-20080318101724-c3ofm3vslli3wfwl  =  -r -5
revid:cvs-1:dann-20080318035819-bwawewmyps2rb2ot    =  -r -11

2635714f3dac5f24eb1997cbf97285810f6799c0  =  HEAD~4
c4d57908d6d8c693e779599182b810565b2eb608  =  HEAD~10


View changes introduced in given revision:

(This shows also the commit message, author and date.)
$ time git show 2635714f3dac5f24eb1997cbf97285810f6799c0 >/dev/null
real    0m0.012s

$ time bzr diff -c revid:cvs-1:wohler-20080318101724-c3ofm3vslli3wfwl >/dev/null
real    2m40.467s


Show differences between two revisions:

$ time git diff HEAD~10..HEAD~4 >/dev/null
real    0m0.076s

$ time bzr diff -r -11..-5 >/dev/null
real    2m44.214s

$ time git diff HEAD~4.. >/dev/null
real    0m0.072s

$ time bzr diff -r -5.. >/dev/null
real    1m21.836s



Creating a branch
-----------------

With git I chose "git checkout -b" instead of "git branch" because the
former also checks out the files as does "bzr branch". The bzr branch is
created inside the same shared repository so that the common objects are
shared.


Create new topic branch based on the head revision of the main
development branch:

$ time git checkout -b topic master >/dev/null
real    0m0.062s

$ time bzr branch trunk topic >/dev/null
real    0m7.249s


Create new topic branch based on earlier revision of main development
branch:

$ time git checkout -b topic master~4 >/dev/null
real    0m0.085s

$ time bzr branch -r -5 trunk topic >/dev/null
real    2m51.551s



Compare branches' commit histories
----------------------------------

In above benchmark I created branch 'topic' which is based on earlier
revision of main development branch. In this test I compared commands
which display commits that are missing from 'topic' branch compared to
the main development branch (four commits in total).


$ time git log topic..master >/dev/null
real    0m0.006s

$ time bzr missing --theirs-only ../trunk >/dev/null
real    18m25.173s



Annotate/blame a file (src/keymap.c)
------------------------------------

$ time git blame src/keymap.c >/dev/null
real    0m8.753s

$ time bzr blame src/keymap.c >/dev/null
real    0m58.296s

Re: Emacs repository benchmark: bzr and git

[Lennart Borgman (gmail)]

Teemu Likonen wrote:
> I did some benchmarking in git and bzr repositories of Emacs. Some
> numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"), 2825
> files. Both repositories seem to have just linear history converted from
> CVS repo. Both have the same head revision which is
> 481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
> revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).
> 
> Repositories/branches are pulled from here:
> git: git://git.sv.gnu.org/emacs.git
> bzr: http://bzr.notengoamigos.org/emacs/trunk/
> 
> My system is AMD Sempron 3000+ with 2 GB memory and it's running Debian
> GNU/Linux 4.0. I'm using the latest development versions of both git
> (1.5.5.rc0.6.gdeda) and bzr (1.4dev). I just measured with 'time'
> command how long it takes to run certain commands.
> 
> 
> 
> Viewing history
> ---------------
> 
> 
> The complete history:
> 
> $ time git log >/dev/null
> real    0m5.741s
> 
> $ time bzr log >/dev/null
> real    3m15.708s

I no nothing about this, but it looks like the bzr developers thinks 
that bzr is as fast as git:

   http://bazaar-vcs.org/Benchmarks

It looks like something is wrong, I have absolutely no idea what. Was 
someone here in contact with the bzr developers?

Re: Emacs repository benchmark: bzr and git

[Matthieu Moy]

Teemu Likonen <tlikonen@iki.fi> writes:

> I just measured with 'time' command how long it takes to run certain
> commands.

Interesting benchmark, but what's missing is whether this is with hot
or cold cache. For example:

> $ time bzr log >/dev/null
> real    3m15.708s

I get a bit less than a minute here, and I don't think my machine is
4x faster than yours, so this was probably with cold cache.

An interesting measure is "best of 3" (run it 3 times, and take the
smallest).

Re: Emacs repository benchmark: bzr and git

[dhruva]

Hi,
 I am using the latest bzr from their repository to access emacs repo.
I am not seeing any performance improvements. If we need to pull from
some other tree for doing performance tests, please let us know. I
even went to #bzr channel on irc.freenode.org to see if someone could
answer my questions, I did not see enough noise or response. Anyway, I
will try again, it might be due to timezone issues...

 To the emacs maintainers and decision makers: What more information
is required to convince bzr is not the right tool at the present
moment? Every tests seems to reveal limitations in the tool compared
to available alternatives and yet I hear some say that the decision
has been made to use bzr. I have not missed out a single discussion
related to this thread and am quite verbosely involved. I personally
do not see a single mail which can tilt the scale in favor of bzr
except it is part of GNU project.

-dhruva

Re: Emacs repository benchmark: bzr and git

[Matthieu Moy]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:

> I no nothing about this, but it looks like the bzr developers thinks
> that bzr is as fast as git:
>
>   http://bazaar-vcs.org/Benchmarks
>
> It looks like something is wrong, I have absolutely no idea what. 

AAUI, the benchmarks were done importing tarballs, not history, and
therefore on a short-size history. I'd say "unrealisticly short-size
history" indeed.

> Was someone here in contact with the bzr developers?

Look carefully at the Cc: list ;-).

-- 
Matthieu

Re: Emacs repository benchmark: bzr and git

[dhruva]

Hi,

On Tue, Mar 18, 2008 at 9:32 PM, Matthieu Moy <Matthieu.Moy@imag.fr> wrote:
> Teemu Likonen <tlikonen@iki.fi> writes:
>
>  > I just measured with 'time' command how long it takes to run certain
>  > commands.
>
>  Interesting benchmark, but what's missing is whether this is with hot
>  or cold cache. For example:
>
>
>  > $ time bzr log >/dev/null
>  > real    3m15.708s
>
>  I get a bit less than a minute here, and I don't think my machine is
>  4x faster than yours, so this was probably with cold cache.
>
>  An interesting measure is "best of 3" (run it 3 times, and take the
>  smallest).
>

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 > :NUL

real    0m30.562s
user    0m0.015s
sys     0m0.000s

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 > :NUL

real    0m34.250s
user    0m0.015s
sys     0m0.015s

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 > :NUL

real    0m33.391s
user    0m0.015s
sys     0m0.000s

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 --short > :NUL

real    0m19.828s
user    0m0.015s
sys     0m0.015s

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 --short > :NUL

real    0m19.390s
user    0m0.015s
sys     0m0.000s

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 --short > :NUL

real    0m18.421s
user    0m0.047s
sys     0m0.031s

dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
$ time bzr log -l 10 --short > :NUL

real    0m18.375s
user    0m0.015s
sys     0m0.000s


Even with 3 runs, I do not see any noticeable change in performance. I
am running the tests on emacs repo. I am running all tests on M$-XP
box (lenovo T61 series with Intel Centrino Pro, 1Gb RAM).

How do I get rid of cache if I have to restart the tests? I plan to
analyze the '--lsprof' output to see if there is different code path
and the so called hot/cold cache making any difference.

-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs repository benchmark: bzr and git

[Andreas Schwab]

Teemu Likonen <tlikonen@iki.fi> writes:

> I did some benchmarking in git and bzr repositories of Emacs. Some
> numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"), 2825
> files. Both repositories seem to have just linear history converted from
> CVS repo. Both have the same head revision which is
> 481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
> revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).

The git repository at savannah has more revisions because it has the
multi-tty arch history imported into it.  It also contains many merges,
although not in recent history.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Stefan Monnier]

>> All of the DVCSs seem good.  No one marshalled any compelling arguments
>> in favor of one versus the other on technical grounds, and all other
>> things being equal, RMS (and maybe some others, perhaps including Yidong
>> and Stefan?) preferred bzr because it is a GNU project.

> In its current form bzr is not usable for a project of the size of Emacs.

I'm currently using Bzr to maintain my private hacking branch (used to
use Arch until now for that).  I use it with Jason's Bzr mirror of the
CVS repository.

The performance is indeed problematic on several operations.
Also the Bzr mirror has some shortcomings currently.

So, currently I'm waiting to see how quickly the problematic performance
problems get resolved and how the mirror's shortcomings get resolved.

I'd also like to hear from people working on other platforms (Macs OS
X, Windows) to make sure that the Bzr mirror can be used there as well.


        Stefan

Re: Emacs repository benchmark: bzr and git

[Teemu Likonen]

Matthieu Moy kirjoitti:

> > $ time bzr log >/dev/null
> > real    3m15.708s
>
> I get a bit less than a minute here, and I don't think my machine is
> 4x faster than yours, so this was probably with cold cache.
>
> An interesting measure is "best of 3" (run it 3 times, and take the
> smallest).

Don't know what I'm doing wrong. I just ran "time bzr log >/dev/null" 
five times in a row and I always get more than three minutes:

3m1.999s
3m6.945s
3m0.548s
3m5.149s
3m5.908s

Re: Emacs repository benchmark: bzr and git

[Stefan Monnier]

>> $ time bzr log >/dev/null
>> real    3m15.708s

> I get a bit less than a minute here, and I don't think my machine is
> 4x faster than yours, so this was probably with cold cache.

I don't think it matters: 1 minute is too long as well.

I don't care if Bzr is slower or faster than Git, but in order to switch
to Bzr, we need it to be "fast enough".  Currently it is not.  At the
very least the "bzr diff -r <something>" should not take more than
a couple seconds and it should be possible to get the "bzr status" of
a file instantaneously.


        Stefan


PS: and now for something completely different: is there a Texinfo
version of the Bazaar reference manual?
PPS: If we want to compare Bzr to Git, we may also want to look at the
repository size: currently, it looks like Git's version of the Emacs
repository weighs in around 200MB, while Bzr's version hovers
around 300MB.

Re: Emacs Bazaar repository

[Juanma Barranquero]

On Tue, Mar 18, 2008 at 5:50 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:

>  I'd also like to hear from people working on other platforms (Macs OS
>  X, Windows) to make sure that the Bzr mirror can be used there as well.

I've done a bit of testing with Jason Earl's mirror, using bzr 1.2.0
on Windows XP.

It is slow. As I already reported,

   bzr log --limit=30 --short lisp\ChangeLog

consumed all available memory and had to be killed. Also surprising,
using Jason's shared repository setup example, a simple

  bzr merge --pull

from the sandbox branch takes ten or more seconds, and that's a purely local op.

 Juanma

Re: Emacs repository benchmark: bzr and git

[Juanma Barranquero]

On Tue, Mar 18, 2008 at 6:43 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:

>  PPS: If we want to compare Bzr to Git, [...]

Oh, but we don't.

 Juanma

Re: Emacs repository benchmark: bzr and git

[Jonathan Corbet]

dhruva <dhruvakm@gmail.com> wrote:

> How do I get rid of cache if I have to restart the tests? I plan to
> analyze the '--lsprof' output to see if there is different code path
> and the so called hot/cold cache making any difference.

Assuming you're running on a Linux kernel:

	# sync
	# echo 3 > /proc/sys/vm/drop_caches

That dumps everything which does not require writeback, leaving you with
a quite cold cache.

jon

Re: Emacs repository benchmark: bzr and git

[Teemu Likonen]

Andreas Schwab kirjoitti:

> Teemu Likonen <tlikonen@iki.fi> writes:
> > I did some benchmarking in git and bzr repositories of Emacs. Some
> > numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"),
> > 2825 files. Both repositories seem to have just linear history
> > converted from CVS repo. Both have the same head revision which is
> > 481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
> > revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).
>
> The git repository at savannah has more revisions because it has the
> multi-tty arch history imported into it.  It also contains many
> merges, although not in recent history.

Oh, I didn't notice that. Indeed, the Emacs bzr repo is 2031 commits 
lighter. In theory this difference seems like a disadvantage to git in 
this test. In practice it's probably not because the length of history 
does not seem to have much effect on git's performance. Anyway, the 
main point was bzr, and git was there just to give some perspective 
(for me personally too).

Re: Emacs Bazaar repository

[Richard Stallman]

This question is over and decided.
We will use GNU Bzr, because it is a GNU package.

Re: Emacs Bazaar repository

[Mike Mattie]

[-- Attachment #1: Type: text/plain, Size: 3096 bytes --]

On Tue, 18 Mar 2008 00:03:09 -0400
Karl Fogel <kfogel@red-bean.com> wrote:

> dhruva <dhruvakm@gmail.com> writes:
> >  With lengthy discussions having taken place on this SCM issue, what
> > are we waiting for? Are there really any concrete plans to move to a
> > dSCM or are we just wasting talking about features and shortcomings
> > of each tool. Any outcome of ESR study?
> 
> I think you may have missed the news :-).
> 
> We decided on bzr, which in practice seems to mean "some people are
> seriously testing bzr, and once we have a good conversion and are
> satisfied we can get work done, someone will install it on savannah
> and that repository will be the new master".
> 
> All of the DVCSs seem good.  No one marshalled any compelling
> arguments in favor of one versus the other on technical grounds, and
> all other things being equal, RMS (and maybe some others, perhaps
> including Yidong and Stefan?) preferred bzr because it is a GNU
> project.

All of them seem good from their press; until you use them and realize
some scale, most don't. When darcs was put forward as a candidate it
was clear that there was far more googling and e-mailing going on
than testing.

Now that people are actually trying them out some reality will filter back in.
You can tell that progress is being made because the strange metaphors, furious
hand-waving, and war stories have been replaced with _numbers_, and some sad faces.

There are three choices from here:

1. Backup, demand real testing and trails, make a new choice. (not very
   political)

2. The choice is already made, so fix bzr. May involve a complete re-design
   if the algorithms/structures are not designed to efficiently process common
   case access patterns (log etc.).

3. Let the issue die quietly because there is not enough man-power to divert
   from Emacs to bzr (who wants to volunteer?), continue using cvs
   indefinitely without rescinding the declaration to use bzr.

The GNU project is hoping for #2 as they want a real contender to git.
The numbers now being produced show why very clearly. 
For the GNU project that may be a very strategic choice in hind-sight
if #2 pans out. 

Nobody has been press-ganged so #2 is really not "terrible",
but Emacs has clearly been volunteered for bzr work, or the conversion
has been postponed indefinitely.

It's not my decision, and I am not making a position here either way.
I just hope putting the three choices in plain print will save some
electricity and mental band-with. 

Anyone who really groks the three options I wrote won't reply, because
it is simply the state of things, and how things turns out is a individual
choice for each developer of how they will spend their time, which is
why GNU does things like #2.

> The above is a summary of what I take to be the current state of
> things. My personal opinions: I'm also happy with bzr, and would be
> equally happy with any of the usual suspects among free software
> DVCS.  I just hope we can switch the master soon.
> 
> -Karl
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> This question is over and decided.
> We will use GNU Bzr, because it is a GNU package.

That is a very bad move.  That will mean that nobody will be able to
effectively work on emacs any more.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs repository benchmark: bzr and git

[James Westby]

[-- Attachment #1: Type: text/plain, Size: 4587 bytes --]

On Tue, 2008-03-18 at 17:43 +0200, Teemu Likonen wrote:
> I did some benchmarking in git and bzr repositories of Emacs. Some
> numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"), 2825
> files. Both repositories seem to have just linear history converted from
> CVS repo. Both have the same head revision which is
> 481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
> revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).
> 

Hi,

Thanks for the concrete numbers that we can work from.

So, I hacked up a quick plugin to output logs in a flat style like
git. I've attached it to this mail. You can install it by dropping
it in to ~/.bazaar/plugins.

It is totally unintegrated with bzr's log framework. I plan on
rectifying that and proposing it for inclusion. It also has
some nasty UI warts like using --end revid to limit the
revisions that you want to see to ones that are not that
revid or the parents of it. This means you can't do things
like bzr flatlog -r-100..

I include the numbers from that inline with yours

> 
> Viewing history
> ---------------
> 
> 
> The complete history:
> 
> $ time git log >/dev/null
> real    0m5.741s
> 
> $ time bzr log >/dev/null
> real    3m15.708s
> 

bzr flatlog > /dev/null  45.17s user 0.75s system 97% cpu 46.883 total

> 
> Last 100 revisions:
> 
> $ time git log -100 >/dev/null
> real    0m0.011s
> 
> $ time bzr log -l100 >/dev/null
> real    2m10.270s
> 

bzr flatlog -l100 > /dev/null  12.11s user 0.32s system 97% cpu 12.714
total

> 
> Last 10 revisions:
> 
> $ time git log -10 >/dev/null
> real    0m0.007s
> 
> $ time bzr log -l10 >/dev/null
> real    2m9.163s
> 

bzr flatlog -l10 > /dev/null  12.18s user 0.25s system 98% cpu 12.582
total


I can also suggest a workaround for anyone that is using bzr and wants
to speed up log further while we work on it.

If you edit ~/.bazaar/bazaar.conf and add

[ALIASES]
flatlog = --end cvs-1:bastien1-20080217010841-op363t09ccs7pais

you can get

bzr flatlog --end cvs-1:bastien1-20080217010841-op363t09ccs7pais
> /dev/null  0.77s user 0.08s system 96% cpu 0.876 total

and still have 1000 revisions to look at.

I'll work on getting -r properly integrated which means you could
have this alias set at all times, and then if you needed full
history you could do

bzr flatlog -r1..

> 
> Creating a branch
> -----------------
> 
> With git I chose "git checkout -b" instead of "git branch" because the
> former also checks out the files as does "bzr branch". The bzr branch is
> created inside the same shared repository so that the common objects are
> shared.

bzr creates a second working tree, git replaces your first.

> 
> 
> Create new topic branch based on the head revision of the main
> development branch:
> 
> $ time git checkout -b topic master >/dev/null
> real    0m0.062s
> 
> $ time bzr branch trunk topic >/dev/null
> real    0m7.249s

to compare getting the second working tree you can use

git clone emacs temp2  1.70s user 1.29s system 31% cpu 9.395 total

(it hardlinks the objects rather than just re-using them as bzr does,
so it's still not the same)

However this is a bit silly, as you are just comparing the usual
ways to get a new branch in that particular VCS.

It is possible to emulate the git way using a couple of bzr commands
if you prefer to work in one directory with one working tree. I don't
have benchmark numbers for that currently though I'm afraid.

> 
> 
> Create new topic branch based on earlier revision of main development
> branch:
> 
> $ time git checkout -b topic master~4 >/dev/null
> real    0m0.085s
> 
> $ time bzr branch -r -5 trunk topic >/dev/null
> real    2m51.551s
> 

There was a patch proposed a couple of days to tackle an efficiency
operation in exactly this command.

> 
> 
> Compare branches' commit histories
> ----------------------------------
> 
> In above benchmark I created branch 'topic' which is based on earlier
> revision of main development branch. In this test I compared commands
> which display commits that are missing from 'topic' branch compared to
> the main development branch (four commits in total).
> 
> 
> $ time git log topic..master >/dev/null
> real    0m0.006s
> 
> $ time bzr missing --theirs-only ../trunk >/dev/null
> real    18m25.173s
> 

An inefficiency was also highlighted here a couple of days ago. I think
something was proposed that we could switch this and other commands
to that would speed them up greatly.

If you want to talk about bzr's performance or features I will be happy
to do so, but please drop the emacs list from the discussion.

Thanks,

James



[-- Attachment #2: flatlog.py --]
[-- Type: text/x-python, Size: 6132 bytes --]



from bzrlib import (
    branch,
    builtins,
    commands,
    errors,
    revision as _mod_revision,
    option,
    osutils,
    )


class cmd_flatlog(commands.Command):
    """Output the log in a flat format."""

    takes_options = [
        option.Option('limit',
                      short_name='l',
                      help='Limit the output to the first N revisions.',
                      argname='N',
                      type=builtins._parse_limit,
                     ),
        option.Option('start',
                      type=str,
                     ),
        option.Option('end',
                      type=str,
                     ),
        option.Option('timezone',
                      type=str,
                      help="Display timezone as local, original or utc",
                     ),
        option.Option('forward',
                      help="Display the revisions from oldest to newest",
                     ),
    ]

    def iter_all_history_topo_sorted(self, repo, revision_id,
            limit=None, end=None, no_merges=False):
        if revision_id in (None, _mod_revision.NULL_REVISION):
            return
        if revision_id == end:
            return
        if limit is not None:
            if limit <= 0:
                raise errors.BzrCommandError("limit option must be positive")
            if limit == 1:
                yield revision_id
                return
        to_process = [revision_id]
        indegree = {revision_id:0}
        graph = repo.get_graph()
        parents_map = {}
        no_merge = True
        _limit = limit
        while to_process:
            node = to_process.pop(0)
            parent_map = graph.get_parent_map([node])
            if node not in parent_map: #ghost
                continue
            parents = parent_map[node]
            parents_map[node] = parents
            for parent in parents:
                if parent == end:
                    continue
                done = indegree.setdefault(parent, 0)
                indegree[parent] += 1
                if done == 0:
                    to_process.append(parent)
        to_process = [revision_id]
        while to_process:
            node = to_process.pop(0)
            parents = parents_map[node]
            if not no_merges or len(parents) < 2:
                yield node
                if limit is not None:
                    limit -= 1
                    if limit == 0:
                        return
            for parent in parents:
                if parent == end or parent is _mod_revision.NULL_REVISION:
                    continue
                if parent in parents_map: #not ghost
                    indegree[parent] -= 1
                    if indegree[parent] == 0:
                        to_process.append(parent)

    def iter_all_history_topo_sorted_forward(self, repo,
            revision_id, end=None, limit=None, no_merges=False):
        if revision_id in (None, _mod_revision.NULL_REVISION):
            return
        if revision_id == end:
            return
        if limit is not None:
            if limit <= 0:
                raise errors.BzrCommandError("limit option must be positive")
        to_process = [revision_id]
        graph = repo.get_graph()
        revs = set()
        while to_process:
            node = to_process.pop()
            parent_map = graph.get_parent_map([node])
            if node not in parent_map: #ghost
                continue
            revs.add(node)
            parents = parent_map[node]
            for parent in parents:
                if parent is not _mod_revision.NULL_REVISION and parent not in revs:
                    to_process.append(parent)
        for rev_id in graph.iter_topo_order(revs):
            if no_merges:
                parent_map = graph.get_parent_map([node])
                if len(parent_map[node]) > 1:
                    continue
            yield rev_id
            if limit is not None:
                limit -= 1
                if limit == 0:
                    return

    def iter_revs(self, repo, start, end, limit, forward, no_merges):
        num = 9
        i = 0
        rev_ids = []
        if forward:
            rev_generator = self.iter_all_history_topo_sorted_forward(repo,
                    start, limit=limit, end=end, no_merges=no_merges)
        else:
            rev_generator = self.iter_all_history_topo_sorted(repo, start,
                limit=limit, end=end, no_merges=no_merges)
        for rev_id in rev_generator:
            rev_ids.append(rev_id)
            i += 1
            if i == num:
                revs = repo.get_revisions(rev_ids)
                for rev in revs:
                    yield rev
                i = 0
                rev_ids = []
                num = min(int(num * 1.5), 200)
        revs = repo.get_revisions(rev_ids)
        for rev in revs:
            yield rev

    def run(self, limit=None, start=None, end=None, timezone=None,
            forward=False, no_merges=False):
        b = branch.Branch.open_containing('.')[0]
        if timezone is None:
            timezone = "original"
        if (start is not None or end is not None) and forward:
            raise errors.BzrCommandError("--forward and --start and --end are not supported yet")
        repo = b.repository
        repo.lock_read()
        try:
            if start is None:
                start = b.last_revision()
            for rev in self.iter_revs(repo, start, end, limit, forward, no_merges):
                self.outf.write("commit %s\nAuthor: %s\nDate:   %s\n\n" % (rev.revision_id,
                            rev.get_apparent_author(), osutils.format_date(rev.timestamp,
                                        rev.timezone or 0, timezone)))
                if not rev.message:
                    self.outf.write("  (no message)\n")
                else:
                    for l in rev.message.split('\n'):
                        self.outf.write("  %s\n" % l)
                self.outf.write("\n")
        finally:
            repo.unlock()


commands.register_command(cmd_flatlog)

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

() Andreas Schwab <schwab@suse.de>
() Tue, 18 Mar 2008 21:05:01 +0100

   That is a very bad move.  That will mean that nobody
   will be able to effectively work on emacs any more.

It's a bad move, but not because nobody will be able to
effectively work on emacs any more (an exaggeration), but
rather because to effectively work on emacs, people will
have to spend time setting up, testing, and debugging
various coping mechanisms.  Big time/effort sink.

On the other hand, if we see something like:

| ttn <-> git <------------> git <-> jrhacker   : high freq
|        emacs              emacs               : changes
|          ^                  ^
|          |                  |
|          v                  v
|         bzr <------------> bzr                : low freq
|        emacs              emacs               : changes

evolve, then maybe that's not so bad.

thi

Re: Emacs Bazaar repository

[Brian Cully]

On 18-Mar-2008, at 15:27, Richard Stallman wrote:
> This question is over and decided.
> We will use GNU Bzr, because it is a GNU package.

	I, for one, welcome another Emacs branch due to politicking. XEmacs  
doesn't cause nearly enough grief already.

-bjc

Re: Emacs repository benchmark: bzr and git

[Eli Zaretskii]

> Date: Tue, 18 Mar 2008 22:03:46 +0530
> From: dhruva <dhruvakm@gmail.com>
> Cc: bazaar@lists.canonical.com, Teemu Likonen <tlikonen@iki.fi>,
> 	emacs-devel@gnu.org
> 
> dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
> $ time bzr log -l 10 --short > :NUL
> 
> real    0m18.421s
> user    0m0.047s
> sys     0m0.031s
> 
> dhruva@dhruva-lxp ~/stub/repo/bzr/emacs/trunk
> $ time bzr log -l 10 --short > :NUL
> 
> real    0m18.375s
> user    0m0.015s
> sys     0m0.000s
> 
> 
> Even with 3 runs, I do not see any noticeable change in performance. I
> am running the tests on emacs repo. I am running all tests on M$-XP
> box (lenovo T61 series with Intel Centrino Pro, 1Gb RAM).
> 
> How do I get rid of cache if I have to restart the tests? I plan to
> analyze the '--lsprof' output to see if there is different code path
> and the so called hot/cold cache making any difference.

I'm just guessing, but judging from your results, it doesn't look like
the times are I/O-bound, and so the cold/hot cache issue is not an
important factor with bzr, at least on MS-Windows.  With I/O-bound
operations, there's a significant difference (factors of 5 to 20)
between the first and the second invocations, which is not the case
here.

Also, I'm guessing that the user/system times are irrelevant because
bzr invokes subsidiary processes that do the actual work (and the
Windows version of system calls used by `time' does not cover time
spent in child processes, unlike the Posix system calls).  That is the
only way I can understand the startling difference between the elapsed
time and the sum of system+user time.  So it's hard to tell where does
bzr spend most of its time from these numbers.

Re: Emacs Bazaar repository

[Mike Mattie]

[-- Attachment #1: Type: text/plain, Size: 434 bytes --]

On Tue, 18 Mar 2008 16:58:37 -0400
Brian Cully <bcully@gmail.com> wrote:

> On 18-Mar-2008, at 15:27, Richard Stallman wrote:
> > This question is over and decided.
> > We will use GNU Bzr, because it is a GNU package.
> 
> 	I, for one, welcome another Emacs branch due to politicking.
> XEmacs doesn't cause nearly enough grief already.

That in itself is a political statement. interesting delimna no ?

> -bjc
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Stefan Monnier]

> This question is over and decided.
> We will use GNU Bzr, because it is a GNU package.

The current performance of Bzr is not good enough when used on the
Emacs repository.

If those performance problems can be resolved in a reasonable amount of
time, fine.  But otherwise, we may have to use some other system, at
least temporarily.


        Stefan

Re: Emacs Bazaar repository

[Jonathan Lange]

On Wed, Mar 19, 2008 at 8:45 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> > This question is over and decided.
>  > We will use GNU Bzr, because it is a GNU package.
>
>  The current performance of Bzr is not good enough when used on the
>  Emacs repository.
>

Concretely, in order to be good enough, which operations need to
improve and by how much?

>  If those performance problems can be resolved in a reasonable amount of
>  time, fine.

Then let's try to do that.

jml

Re: Emacs Bazaar repository

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> This question is over and decided.
>> We will use GNU Bzr, because it is a GNU package.
>
> The current performance of Bzr is not good enough when used on the
> Emacs repository.
>
> If those performance problems can be resolved in a reasonable amount of
> time, fine.  But otherwise, we may have to use some other system, at
> least temporarily.

We could, y'know, stick with CVS.

Re: Emacs Bazaar repository

[Nick Roberts]

 > >> This question is over and decided.
 > >> We will use GNU Bzr, because it is a GNU package.
 > >
 > > The current performance of Bzr is not good enough when used on the
 > > Emacs repository.
 > >
 > > If those performance problems can be resolved in a reasonable amount of
 > > time, fine.  But otherwise, we may have to use some other system, at
 > > least temporarily.
 > 
 > We could, y'know, stick with CVS.

This is, indeed, the only solution within the given constraints: we use CVS
until Bzr meets the performance requirements for use with the Emacs repository.
I certainly don't want to learn how to use a third VCS and, in practice, if
we used one temporarily, people would be reluctant to move away from it.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: Emacs Bazaar repository

[Johannes Weiner]

Hi,

Richard Stallman <rms@gnu.org> writes:

> This question is over and decided.
> We will use GNU Bzr, because it is a GNU package.

Wow, that sounds almost democratic.

	Hannes

Re: Emacs Bazaar repository

[Thomas Lord]

Chong Yidong wrote:
> We could, y'know, stick with CVS.
>   


I'm not so sure that that's a crazy idea.   

You could add in a couple of shell scripts to do
fancy merging tricks and keep track of the history
of "patches" submitted and applied, keeping all of
that new meta-data under CVS.

Contributors could use git if they want, as long as
they don't mind using some of those scripts to prepare
patches to submit.

You get the "dvcs" effect that way, without changing
the vcs....

If it's done in simple and clean ways then people
could add on features like web browsing interfaces
and "log" reporting features.


-t




>
>
>   

Re: Emacs Bazaar repository

[Mike Mattie]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

On Tue, 18 Mar 2008 17:13:23 -0700
Thomas Lord <lord@emf.net> wrote:

> Chong Yidong wrote:
> > We could, y'know, stick with CVS.
> >   
> 
> 
> I'm not so sure that that's a crazy idea.   
> 
> You could add in a couple of shell scripts to do
> fancy merging tricks and keep track of the history
> of "patches" submitted and applied, keeping all of
> that new meta-data under CVS.
> 
> Contributors could use git if they want, as long as
> they don't mind using some of those scripts to prepare
> patches to submit.
> 
> You get the "dvcs" effect that way, without changing
> the vcs....
> 
> If it's done in simple and clean ways then people
> could add on features like web browsing interfaces
> and "log" reporting features.

If you were going to invest your time and skills into hacking on a VCS do you
really want to choose coding out-of-tree scripts for CVS ?

Any ideas that would retro-fit DVCS features onto archaic VCS systems
would be more effective for Emacs in vc I think. From there you could
at least expect a normalized API for common VCS operations and a larger
audience for your features.

It seems better to work on a better future for a modern VCS system, all things equal,
than working on arranging the flowers at CVS's funeral.

Your analysis from the Shift Selection thread is meticulous IMHO, this must be
a floater (idea).

btw, I have coded some large shell scripts regretfully. Even basic string operations are
     not simple ; hence perl. I doubt that any fancy merging tricks can emerge in sane 
     form without writing a FOO -> bash compiler.

     The auto-tools suite is essentially useful, but I doubt anyone is thrilled by
     maintaining autoconf.

     even if you get daring and use co-processes to leverage an external "to bash"
     translator to feedback complex semantics into your bash core, many, 
     if not most programs cannot read from pipes without SIGPIPE. The mmap assumption
     is almost universal these days.

     If you need any more horror stories along these lines, I will share them privately :)

> 
> -t
> 
> 
> 
> 
> >
> >
> >   
> 
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Thomas Lord]

Mike Mattie wrote:
> If you were going to invest your time and skills into hacking on a VCS do you
> really want to choose coding out-of-tree scripts for CVS ?
>   

The question of making short-term investments of my time
in this area has been weighing heavily on my mind, lately.
The original Arch took like 4-6 weeks to code and, now, years
later, I know a lot more.   So I'm trying to think of what I might
do in this area.

I don't think I want to make any programs that are CVS-specific
in any way  -- but I am thinking that the "dcvs" needs of a project
like Emacs can possibly be done in a way that doesn't make it
necessary to first move away from CVS.   Desirable to eventually
move away, absolutely -- but i'm not sure that good dcvs features
need to force that migration to happen right away.



> Any ideas that would retro-fit DVCS features onto archaic VCS systems
> would be more effective for Emacs in vc I think. From there you could
> at least expect a normalized API for common VCS operations and a larger
> audience for your features.
>
> It seems better to work on a better future for a modern VCS system, all things equal,
> than working on arranging the flowers at CVS's funeral.
>
>   

Sure.  I'm not proposing to invest in CVS' future.   I'm suggesting
working on dvcs needs with the constraint that it's desirable to meet
dcvs needs without *forcing* a migration away from CVS (or any
other system).   It's best, to the extent possible, for dvcs features to
be independent of the vcs system used by each programmer.



> Your analysis from the Shift Selection thread is meticulous IMHO, this must be
> a floater (idea).
>
>   

The VCS stuff is, *yes* a "floater" speculative idea.   The shift-select 
stuff
is, ahem, well... as you say, thank you.


I'm sketchy about the concept of trying to muster volunteer labor.

But I'm a little bit hooked in interest on the dcvs woes here.

I'm just wondering what makes sense to try to do here.   Whether
I can mount a quick and dirty "Arch 2.0" push to any good effect.

-t




> btw, I have coded some large shell scripts regretfully. Even basic string operations are
>      not simple ; hence perl. I doubt that any fancy merging tricks can emerge in sane 
>      form without writing a FOO -> bash compiler.
>
>      The auto-tools suite is essentially useful, but I doubt anyone is thrilled by
>      maintaining autoconf.
>
>      even if you get daring and use co-processes to leverage an external "to bash"
>      translator to feedback complex semantics into your bash core, many, 
>      if not most programs cannot read from pipes without SIGPIPE. The mmap assumption
>      is almost universal these days.
>
>      If you need any more horror stories along these lines, I will share them privately :)
>
>   
>> -t
>>
>>
>>
>>
>>     
>>>   
>>>       
>>
>>     

Re: Emacs Bazaar repository

[Stefan Monnier]

>> > This question is over and decided.
>> > We will use GNU Bzr, because it is a GNU package.
>> The current performance of Bzr is not good enough when used on the
>> Emacs repository.

> Concretely, in order to be good enough, which operations need to
> improve and by how much?

I've posted those I bumped into and consider serious, as bug reports on
the Bzr bug tracker.


        Stefan

Re: Emacs repository benchmark: bzr and git

[dhruva]

Hi,

On Wed, Mar 19, 2008 at 12:11 AM, Jonathan Corbet <corbet@lwn.net> wrote:
> dhruva <dhruvakm@gmail.com> wrote:
>
>  > How do I get rid of cache if I have to restart the tests? I plan to
>  > analyze the '--lsprof' output to see if there is different code path
>  > and the so called hot/cold cache making any difference.
>
>  Assuming you're running on a Linux kernel:

I am on M$ (XP). So, the cache that has been referred so far is the
operating system virtual memory manager caching and not something
specific to bzr. If this is the case, then the same caching concepts
will work for any application using VM. Is bzr coded differently to
make use of such a feature in a different way? I am interested in
exploring this to find out if it can help performance or just a myth.

-dhruva

Re: Emacs repository benchmark: bzr and git

[Richard Stallman]

     To the emacs maintainers and decision makers: What more information
    is required to convince bzr is not the right tool at the present
    moment?

This decision is not a decision for the present moment.  It is a long
term decision.

So it would be better to wait a few months while Bzr developers
improve it, than to make some other "temporary" decision that would
probably be hard to reverse.

Re: Emacs Bazaar repository

[Bastien]

dhruva <dhruvakm@gmail.com> writes:

> I just hope it the decision is based on technical facts rather than
> affiliations and emotions...

I think the right dimension to consider is the social one.

The assumption in the discussion so far has been namely this one: it is
possible to use a dVCS instead of the current CVS without switching to a
new project workflow.  Even better: most dVCS are so flexible that you
can adapt your project workflow _after_ you start using one of them.

This has been specifically stated in Stephen J. Turnbull's post (from
January 08):

  http://article.gmane.org/gmane.emacs.devel/86530

> Thus I suggest that the project workflow discussion be postponed until
> the core stakeholders are satisfied that the new tools are functioning
> stably in the current workflow.  This will take only a month, at most
> two, once the tools are chosen.  As you point out, a big advantage of
> a dVCS is the flexibility of the workflow even after it is installed.
> There is no big loss to waiting a bit, and a potential large gain: the
> improved understanding of the tools that the core stakeholders will
> have.

... and this is clearly *true* when the use of the dVCS system is not
problematic /per se/.

Now we are in a situation where the decision for the dVCS has been made
(bzr) and yet this decision is not satisfactory for many of us. 

So instead of trying to convince 120 developpers to use bzr in its
current unsatisfactory state, maybe it's time to open the discussion
about the project workflow, and see if the number of people to convince
cannot be reduced by such a discussion.

Now referring to Gregory Collins' post[1] about the different workflows,
I think the current workflow for Emacs would be translated into a mix of
the "star" topology (where many developers are entitled to write in any
part of the codebase) and the "Lieutenant" topology (where the majority
of developers are responsible for subsystems.)

This would mean that some core developers have access to all the code,
and need to agree about when bzr is good enough, and other developers
are free to use whatever they want, provided that they send patches in
the form that the core developers prefer.

I guess at most 25% of the current 120 developers would need to have
access to all the code.  The rest would be responsible for a limited
part -- either a large directory like Gnus or a single Elisp file.

Of course, it would be easier if all the Lieutenants were using bzr,
because the core team would just need to pull from the lieutenants
repositories.  But the only true requisit is that lieutenants keep
pulling from the Emacs codebase and make sure the code they are
responsible for works correctly with the latest Emacs code.

We spent time learning that the radical change with dVCS is that 
they move the constraints for the workflow from the technical to 
the social dimension.  It's strange that the discussion about bzr 
is just a technical one, without consideration about the social 
change this already calls for.

Notes: 
[1]  http://article.gmane.org/gmane.emacs.devel/86503

-- 
Bastien

Re: Emacs Bazaar repository

[Juanma Barranquero]

On Wed, Mar 19, 2008 at 6:44 AM, Bastien <bzg@altern.org> wrote:

>  The assumption in the discussion so far has been namely this one: it is
>  possible to use a dVCS instead of the current CVS without switching to a
>  new project workflow.  Even better: most dVCS are so flexible that you
>  can adapt your project workflow _after_ you start using one of them.

This is one assumption. The other one is that all dVCS (or at least,
the three or four "major" ones) are largely equivalent. Both seem to
be false at this moment.

>  This would mean that some core developers have access to all the code,
>  and need to agree about when bzr is good enough, and other developers
>  are free to use whatever they want, provided that they send patches in
>  the form that the core developers prefer.
>
>  I guess at most 25% of the current 120 developers would need to have
>  access to all the code.  The rest would be responsible for a limited
>  part -- either a large directory like Gnus or a single Elisp file.

I don't think I agree. Until now the Emacs model has been that
everyone with write access is trusted to display some common sense,
and it has worked quite well. Limiting who can write to the canonical
repository and establishing that kind of hierarchy seems like fixing a
non-existent problem. As you say, it is a social issue. If I do a
pervasive change in ido or cua, I *will* send it to Kim, for example;
but I don't want to have to pass through someone to fix a typo or
correct a silly oversight. Let's not create bureaucracy until we know
we need it.

>  It's strange that the discussion about bzr
>  is just a technical one

Which technical discussion?

 Juanma

Re: Emacs repository benchmark: bzr and git

[Teemu Likonen]

Matthieu Moy kirjoitti:

> "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:
> > I no nothing about this, but it looks like the bzr developers
> > thinks that bzr is as fast as git:
> >
> >   http://bazaar-vcs.org/Benchmarks
> >
> > It looks like something is wrong, I have absolutely no idea what.
>
> AAUI, the benchmarks were done importing tarballs, not history, and
> therefore on a short-size history. I'd say "unrealisticly short-size
> history" indeed.

The page says:

"This page shows how Bazaar stacks up on 40-50 real Use Cases on 33 real 
projects."

<http://bazaar-vcs.org/Benchmarks>

If the benchmarks were done without any VCS history at all, then it 
really is not about "real Use Cases" on "real projects" as it claims.

Re: Emacs Bazaar repository

[Sascha Wilde]

dhruva <dhruvakm@gmail.com> wrote:
> If mercurial decides to support multiple branches under the same
> folder, I wonder what impact it might have on the speed.

FWIW it does.
See http://www.selenic.com/mercurial/wiki/index.cgi/Branch

cheers
sascha
-- 
Sascha Wilde

"There is no reason why anyone would want a computer in their home"
Ken Olson, DEC, 1977

Re: Emacs repository benchmark: bzr and git (rerun)

[Teemu Likonen]

Teemu Likonen kirjoitti:

> I did some benchmarking in git and bzr repositories of Emacs. Some
> numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"),
> 2825 files. Both repositories seem to have just linear history
> converted from CVS repo. Both have the same head revision which is
> 481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
> revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).
>
> Repositories/branches are pulled from here:
> git: git://git.sv.gnu.org/emacs.git
> bzr: http://bzr.notengoamigos.org/emacs/trunk/
>
> My system is AMD Sempron 3000+ with 2 GB memory and it's running
> Debian GNU/Linux 4.0. I'm using the latest development versions of
> both git (1.5.5.rc0.6.gdeda) and bzr (1.4dev). I just measured with
> 'time' command how long it takes to run certain commands.


Hey! I realised that the emacs bzr repository was not fully optimized 
with "bzr pack" command. By running this command it really improves the 
performance of bzr; now I'm getting similar numbers as some of you. I 
really apologise for the misleading information I have spread. I want 
to correct my mistakes by running my tests again (see below).

This raises questions though. I had downloaded the premade emacs bzr 
repo from <http://bzr.notengoamigos.org/> and it seems to have its repo 
pretty much optimized since it performs much better than my previous 
benchmark. I had done almost nothing with the repository after that, 
just some bzr-pulls and performance tests. How come the emacs bzr 
repository slows down so much and so quickly? My experience is that you 
definitely want to run "bzr pack" quite often.

Anyway, here are the same tests with bzr-packed and git-gc'd 
repositories. This test shows that both have improved their 
performance, especially bzr. I did run all the tests three times to see 
if caches have effect. They didn't: all the three runs gave very much 
the same numbers. I picked up the best one anyway.

Again, I'm really sorry about my previous tests.


> Viewing history
> ---------------

I want to point out that in git you can always see the log 
_immediately_, no matter how long it takes to display the whole thing. 
In bzr these numbers reflect pretty much the time to get anything 
visible at all after entering the command.


> The complete history:
>
> $ time git log >/dev/null
> real    0m5.741s
>
> $ time bzr log >/dev/null
> real    3m15.708s

git: 0m3.348s
bzr: 1m15.143s


> Last 100 revisions:
>
> $ time git log -100 >/dev/null
> real    0m0.011s
>
> $ time bzr log -l100 >/dev/null
> real    2m10.270s

git: 0m0.009s
bzr: 0m26.562s


> Last 10 revisions:
>
> $ time git log -10 >/dev/null
> real    0m0.007s
>
> $ time bzr log -l10 >/dev/null
> real    2m9.163s

git: 0m0.005s
bzr: 0m25.519s


> The complete history of a single file:
>
> $ time git log src/keymap.c >/dev/null
> real    0m9.240s
> (The same as above but with detecting and following possible
> renames:) $ time git log --follow src/keymap.c >/dev/null
> real    0m17.891s
>
> $ time bzr log src/keymap.c >/dev/null
> real    3m35.431s

git: 0m5.127s
git: 0m8.953s (with --follow)
bzr: 0m55.461s


> Differences between revisions
> -----------------------------

> View changes introduced in given revision:
>
> (This shows also the commit message, author and date.)
> $ time git show 2635714f3dac5f24eb1997cbf97285810f6799c0 >/dev/null
> real    0m0.012s
>
> $ time bzr diff -c revid:cvs-1:wohler-20080318101724-c3ofm3vslli3wfwl
> >/dev/null real    2m40.467s

git: 0m0.010s
bzr: 0m23.315s


> Show differences between two revisions:
>
> $ time git diff HEAD~10..HEAD~4 >/dev/null
> real    0m0.076s
>
> $ time bzr diff -r -11..-5 >/dev/null
> real    2m44.214s

git: 0m0.068s
bzr: 0m24.140s


> $ time git diff HEAD~4.. >/dev/null
> real    0m0.072s
>
> $ time bzr diff -r -5.. >/dev/null
> real    1m21.836s

git: 0m0.060s
bzr: 0m12.889s


> Creating a branch
> -----------------
>
> With git I chose "git checkout -b" instead of "git branch" because
> the former also checks out the files as does "bzr branch". The bzr
> branch is created inside the same shared repository so that the
> common objects are shared.
>
>
> Create new topic branch based on the head revision of the main
> development branch:
>
> $ time git checkout -b topic master >/dev/null
> real    0m0.062s
>
> $ time bzr branch trunk topic >/dev/null
> real    0m7.249s

git: 0m0.043s
bzr: 0m4.504s


> Create new topic branch based on earlier revision of main development
> branch:
>
> $ time git checkout -b topic master~4 >/dev/null
> real    0m0.085s
>
> $ time bzr branch -r -5 trunk topic >/dev/null
> real    2m51.551s

git: 0m0.062s
bzr: 0m30.120s


> Compare branches' commit histories
> ----------------------------------
>
> In above benchmark I created branch 'topic' which is based on earlier
> revision of main development branch. In this test I compared commands
> which display commits that are missing from 'topic' branch compared
> to the main development branch (four commits in total).
>
>
> $ time git log topic..master >/dev/null
> real    0m0.006s
>
> $ time bzr missing --theirs-only ../trunk >/dev/null
> real    18m25.173s

git: 0m0.004s
bzr: 13m48.239s


> Annotate/blame a file (src/keymap.c)
> ------------------------------------
>
> $ time git blame src/keymap.c >/dev/null
> real    0m8.753s
>
> $ time bzr blame src/keymap.c >/dev/null
> real    0m58.296s

git: 0m7.954s
bzr: 0m17.114s

Re: Emacs repository benchmark: bzr and git (rerun)

[James Westby]

On Wed, 2008-03-19 at 13:37 +0200, Teemu Likonen wrote:
> Hey! I realised that the emacs bzr repository was not fully optimized 
> with "bzr pack" command. By running this command it really improves the 
> performance of bzr; now I'm getting similar numbers as some of you. I 
> really apologise for the misleading information I have spread. I want 
> to correct my mistakes by running my tests again (see below).
> 
> This raises questions though. I had downloaded the premade emacs bzr 
> repo from <http://bzr.notengoamigos.org/> and it seems to have its repo 
> pretty much optimized since it performs much better than my previous 
> benchmark. I had done almost nothing with the repository after that, 
> just some bzr-pulls and performance tests. How come the emacs bzr 
> repository slows down so much and so quickly? My experience is that you 
> definitely want to run "bzr pack" quite often.

Hi,

Thanks for updating us.

I wanted to point out that bzr automatically repacks on commit
every so often, this means that the performance degredation should
be bounded, but yes, you will often be able to speed it up by
running "pack" manually.

In recent versions (I don't know which exactly, sorry) git has
also added the auto-repack on commit. I don't know how their
strategy differs from bzr's, but they should both ensure that
it never gets really bad.

I do think the effects of repacking bzr are very drastic, so it
is probably worth investigating where the slowdown was.

Do you still have your .bzr/repository/ from the old tests?
Is there anything in .bzr/repository/obsolete_packs/ ? That would
let us know just how many packs you were running with.

Thanks,

James

Re: Emacs Bazaar repository

[Bastien]

"Juanma Barranquero" <lekktu@gmail.com> writes:

>>  The assumption in the discussion so far has been namely this one: it is
>>  possible to use a dVCS instead of the current CVS without switching to a
>>  new project workflow.  Even better: most dVCS are so flexible that you
>>  can adapt your project workflow _after_ you start using one of them.
>
> This is one assumption. The other one is that all dVCS (or at least,
> the three or four "major" ones) are largely equivalent. Both seem to
> be false at this moment.

They are not equivalent in terms of performance, but I think the
assumption was more precisely that all dVCS are equivalent in terms of
their ability to adapt to any workflow.  This assumption is true IMO.

> Until now the Emacs model has been that everyone with write access is
> trusted to display some common sense, and it has worked quite well.

Sure, thanks to people on this mailing list!

> Limiting who can write to the canonical repository and establishing
> that kind of hierarchy seems like fixing a non-existent problem.

I didn't want to fix problems in the current workflow.

One of the benefits of using a dVCS is that you can envision different
workflows.  If everybody were okay with bzr then there would be no point
in trying to imagine other workflows before using the new dVCS.  But
since bzr has some annoying shortcomings, maybe it is useful to be sure
that everyone wants to stick to the current workflow before avoiding bzr
because of its inability to preserve this workflow...  

One of the shortcomings of the current workflow is this one: some piece
of code in Emacs is actively developped outside of the Emacs CVS (Gnus,
ERC, Org, etc.)  Since these pieces are also part of Emacs, any change
on them in the Emacs CVS requires someone to report the changes in the
local, independant repository of the module.  This is double work, and
such energy could be spared with the "Lieutenant" topology previously
described.

> As you say, it is a social issue. If I do a pervasive change in ido or
> cua, I *will* send it to Kim, for example; but I don't want to have to
> pass through someone to fix a typo or correct a silly oversight. 

I don't propose to sparse the code into areas where only one developer
can make changes.  I propose to think about what a _distributed_ VCS
would be really useful for as a collective tool, in hope that such a
discussion might give directions in the evaluation of bzr.

(Of course, a dVCS is already useful from an individual point of view:
being able to commit locally, to work on several local branches, all
this means it's already worth using a dVCS.  But bzr might also be
useful as a collective tool.)

> Let's not create bureaucracy until we know we need it.

Well, I believe the real benefit of a dVCS is precisely to avoid
bureaucracy -- provided that you can adapt the tool to the way people
want to work together with it, and this requires some discussion.

-- 
Bastien

Re: Emacs Bazaar repository

[Juanma Barranquero]

On Wed, Mar 19, 2008 at 3:04 PM, Bastien <bzg@altern.org> wrote:

>  They are not equivalent in terms of performance, but I think the
>  assumption was more precisely that all dVCS are equivalent in terms of
>  their ability to adapt to any workflow.  This assumption is true IMO.

Performance often influences workflow.

Also, that every dVCS is flexible enough to adapt does not mean that
all workflows are equally practical or easy to follow with any of
them. Were this not the Emacs project, determining the workflow and
choosing the tool that best matches it would seem preferable to trying
to shoehorn this or that tool to the desired workflow.

>  One of the benefits of using a dVCS is that you can envision different
>  workflows.  If everybody were okay with bzr then there would be no point
>  in trying to imagine other workflows before using the new dVCS.  But
>  since bzr has some annoying shortcomings, maybe it is useful to be sure
>  that everyone wants to stick to the current workflow before avoiding bzr
>  because of its inability to preserve this workflow...

I don't think we're at that point yet. We're at the point were normal
operations are slow because of the Emacs' repository size and long
history. At least, that's what it seems from the comments so far.

>  One of the shortcomings of the current workflow is this one: some piece
>  of code in Emacs is actively developped outside of the Emacs CVS (Gnus,
>  ERC, Org, etc.)  Since these pieces are also part of Emacs, any change
>  on them in the Emacs CVS requires someone to report the changes in the
>  local, independant repository of the module.  This is double work, and
>  such energy could be spared with the "Lieutenant" topology previously
>  described.

Agreed. But there are relatively few, very specific parts of Emacs
that suffer this problem: gnus, org, etc.

>  I propose to think about what a _distributed_ VCS
>  would be really useful for as a collective tool, in hope that such a
>  discussion might give directions in the evaluation of bzr.

That kind of discussion should be independent of the tool, shouldn't it?

 Juanma

Re: Emacs Bazaar repository

[Bastien]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> Also, that every dVCS is flexible enough to adapt does not mean that
> all workflows are equally practical or easy to follow with any of
> them.

True.

> Were this not the Emacs project, determining the workflow and choosing
> the tool that best matches it would seem preferable to trying to
> shoehorn this or that tool to the desired workflow.

My guess is that the collective workflow will change after the switch.  
My point was to try thinking about such possible changes, depending on
what people think of the current workflow. 

> I don't think we're at that point yet. We're at the point were normal
> operations are slow because of the Emacs' repository size and long
> history. At least, that's what it seems from the comments so far.

Okay.  Hopefully bzr will improve quickly.

> Agreed. But there are relatively few, very specific parts of Emacs
> that suffer this problem: gnus, org, etc.

Right.  But think of the work on the rmail-mbox-branch.  If this branch
was really an independant subproject (like Gnus), it would help people
jumping in.  

>>  I propose to think about what a _distributed_ VCS
>>  would be really useful for as a collective tool, in hope that such a
>>  discussion might give directions in the evaluation of bzr.
>
> That kind of discussion should be independent of the tool, shouldn't
> it?

Yes, sure.  But it seems more useful to have this discussion instead of
the git-bzr-mercurial debate...  

-- 
Bastien

Re: Emacs Bazaar repository

[Nicholas Allen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
Have you submitted a bug report?

Nick

Juanma Barranquero wrote:
| On Fri, Mar 14, 2008 at 8:16 AM, Dan Nicolaescu <dann@ics.uci.edu> wrote:
|
|>  The content is OK, but it takes 30 seconds to run
|>  bzr log --limit=30 --short FILENAME
|
| On Windows XP (with prebuilt binary of 1.2.0), the command
|
|   bzr log --limit=30 --short lisp\ChangeLog
|
| brought pagefile size up from ~500 MB (before running bazaar) to
|> 2.5GB (and I had to kill it). It is repeatable.
|
|  Juanma
|
| -------------------------
|

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iD8DBQFH4T/NbpmWsXfOU58RAribAJ9i5Mcs7nD3IBBNsh3ht3U5mq1XUgCfT2HG
pyFXC1fgEZlgW69JaMVr174=
=1Fjm
-----END PGP SIGNATURE-----

Re: Emacs repository benchmark: bzr and git (rerun)

[Teemu Likonen]

Teemu Likonen kirjoitti:

> Hey! I realised that the emacs bzr repository was not fully optimized
> with "bzr pack" command. By running this command it really improves
> the performance of bzr; now I'm getting similar numbers as some of
> you. I really apologise for the misleading information I have spread.
> I want to correct my mistakes by running my tests again (see below).
>
> This raises questions though. I had downloaded the premade emacs bzr
> repo from <http://bzr.notengoamigos.org/> and it seems to have its
> repo pretty much optimized since it performs much better than my
> previous benchmark. I had done almost nothing with the repository
> after that, just some bzr-pulls and performance tests. How come the
> emacs bzr repository slows down so much and so quickly? My experience
> is that you definitely want to run "bzr pack" quite often.

I know the pattern now. The emacs bzr repo gets quite much slower after 
every "bzr pull" update. My second test (the rerun) in bzr-packed 
repository gave 1 minute 15 seconds for whole-history "bzr log". Since 
then I have bzr-pulled three times (20 new commits in total) and I get 
1 minute 45 seconds for "bzr log" now. Just a couple of pulls more 
without packing the repo and we'd be back in the situation of my first 
benchmarks (i.e. "bzr log" taking three minutes).

Re: Emacs repository benchmark: bzr and git (rerun)

[Robert Collins]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

On Wed, 2008-03-19 at 12:17 +0000, James Westby wrote:
> 
> I do think the effects of repacking bzr are very drastic, so it
> is probably worth investigating where the slowdown was.

I think its the same thing that John, Aaon and I have been discussing
about index miss costs; once that is fixed the degredation on local disk
from multiple packs should be significantly reduced.

-Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Jonathan Lange]

On Wed, Mar 19, 2008 at 7:05 AM, Andreas Schwab <schwab@suse.de> wrote:
> Richard Stallman <rms@gnu.org> writes:
>
>  > This question is over and decided.
>  > We will use GNU Bzr, because it is a GNU package.
>
>  That is a very bad move.  That will mean that nobody will be able to
>  effectively work on emacs any more.
>

I don't know about that.

I hacked the README file and replaced every 'e' with an 'o' (emulating
a common typo for me). cvs diff takes over 5 seconds, Bazaar takes
under 1. It took me a lot longer to get the Bazaar repository than it
did to get the Emacs working tree, but now that I have it, lots of
basic operations are faster.

My CVS skills are really, really rusty, so this might not be a fair
comparison, but 'bzr status' takes about 0.3 seconds while 'cvs
status' takes about 8 seconds. Also, although this is a bit
subjective, I find the output of 'bzr status' a lot more helpful for
quickly assessing what I've changed.

Most working days, I really only[1] use 'status', 'diff', 'merge',
'branch', 'revert', 'commit' and basic file ops like 'add' and 'mv'.
These benefit greatly from local access.

jml

[1] Well, those are the primitives I use. I also frequently use
'shelve' and 'unshelve' from the rather sweet bzrtools plugin.

Re: Emacs Bazaar repository

[Stefan Monnier]

>> I'd also like to hear from people working on other platforms (Macs OS
>> X, Windows) to make sure that the Bzr mirror can be used there as well.

> I've done a bit of testing with Jason Earl's mirror, using bzr 1.2.0
> on Windows XP.

> It is slow. As I already reported,

>    bzr log --limit=30 --short lisp\ChangeLog

> consumed all available memory and had to be killed. Also surprising,
> using Jason's shared repository setup example, a simple

>   bzr merge --pull

> from the sandbox branch takes ten or more seconds, and that's a purely local op.

I was not talking about performance (we know about those problems
already).  I'm expecting problems such as "doesn't build", e.g. because
of EOL issues.


        Stefan

Re: Emacs Bazaar repository

[Jason Rumney]

Stefan Monnier wrote:

> I was not talking about performance (we know about those problems
> already).  I'm expecting problems such as "doesn't build", e.g. because
> of EOL issues.
>   

bzr doesn't seem to mess with EOLs, so the build runs fine.

Re: Emacs Bazaar repository

[Stefan Monnier]

>> I was not talking about performance (we know about those problems
>> already).  I'm expecting problems such as "doesn't build", e.g. because
>> of EOL issues.

> bzr doesn't seem to mess with EOLs, so the build runs fine.

I don't see why the first implies the second, but I'm glad to hear it
builds correctly.


        Stefan

Re: Emacs Bazaar repository

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 20 Mar 2008 11:33:35 -0400
> Cc: Juanma Barranquero <lekktu@gmail.com>, Emacs Devel <emacs-devel@gnu.org>
> 
> >> I was not talking about performance (we know about those problems
> >> already).  I'm expecting problems such as "doesn't build", e.g. because
> >> of EOL issues.
> 
> > bzr doesn't seem to mess with EOLs, so the build runs fine.
> 
> I don't see why the first implies the second

Because the files in their original form build just fine.  Changing
the EOL format changes the files, which gives birth to the possibility
that the messed up files will prevent a build.

Re: Emacs Bazaar repository

[dhruva]

Hi,

On Thu, Mar 20, 2008 at 7:30 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>  I was not talking about performance (we know about those problems
>  already).  I'm expecting problems such as "doesn't build", e.g. because
>  of EOL issues.

I face problems when I network connectivity is choppy. I get a crash
when I lose network connection in between a 'bzr pull'. Every time I
see the traceback, I fear corrupting my repository and having to do a
clone all over again. It unfortunately comes back to performance. Had
it been fast, I will have done another clone and compared the folders
or something like that.
 From a feature perspective, bzr is mature enough to be able to use in
big projects. Just the performance bottlenecks makes it difficult to
accept it.
 I hear that there are some patches for bzr to make it fast. I request
the bzr developers to create a bzr branch with performance
improvements that we can just pull and use.

-dhruva

-- 
Contents reflect my personal views only!

Re: Emacs Bazaar repository

[Michael Olson]

[-- Attachment #1: Type: text/plain, Size: 1170 bytes --]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> On the other hand, if we see something like:
>
> | ttn <-> git <------------> git <-> jrhacker   : high freq
> |        emacs              emacs               : changes
> |          ^                  ^
> |          |                  |
> |          v                  v
> |         bzr <------------> bzr                : low freq
> |        emacs              emacs               : changes
>
> evolve, then maybe that's not so bad.

I, for one, will be using git instead of bzr to make initial changes to
Emacs.  Once tested, my changes will then be imported from git to bzr by
means of some script, much like my current setup for the git emacs repo
at git.sv.gnu.org and CVS HEAD.

Once I learned that git branches were simply pointers into a DAG, and
once I experienced the absurdly good speed of git, there was simply no
going back to bzr and Mercurial.

-- 
|       Michael Olson  |  FSF Associate Member #652     |
| http://mwolson.org/  |  Hobbies: Lisp, HCoop          |
| Projects: Emacs, Muse, ERC, EMMS, ErBot, DVC, Planner |
`-------------------------------------------------------'

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

() Michael Olson <mwolson@gnu.org>
() Fri, 21 Mar 2008 21:23:58 -0700

   I, for one, will be using git instead of bzr to make initial
   changes to Emacs.  Once tested, my changes will then be
   imported from git to bzr by means of some script, much like my
   current setup for the git emacs repo at git.sv.gnu.org and CVS
   HEAD.

Could you post the scripts or a link to them?
I'd like to try out the setup.

thi

Re: Emacs Bazaar repository

[Stefan Monnier]

> After a few false starts I have been successful (at least as far as I
> can tell) in importing the Emacs CVS repository into Bazaar.  If anyone
> is interested in playing with the results you can check out the HEAD of
> the CVS repository with:

> bzr clone http://bzr.notengoamigos.org/emacs/trunk/

Other than the tags (which it seems are still missing), it seems that an
important missing information is the rename info: is cvsps unable to
recognize file renames?


        Stefan

Re: Emacs Bazaar repository

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Other than the tags (which it seems are still missing), it seems that an
> important missing information is the rename info: is cvsps unable to
> recognize file renames?

cvsps only computes changesets, nothing more.  git computes renames on
the fly, btw.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs Bazaar repository

[Michael Olson]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Michael Olson <mwolson@gnu.org>
> () Fri, 21 Mar 2008 21:23:58 -0700
>
>    I, for one, will be using git instead of bzr to make initial
>    changes to Emacs.  Once tested, my changes will then be
>    imported from git to bzr by means of some script, much like my
>    current setup for the git emacs repo at git.sv.gnu.org and CVS
>    HEAD.
>
> Could you post the scripts or a link to them?
> I'd like to try out the setup.

Here is a write-up explaining how I do all this:
<http://mwolson.org/projects/GitCvsSync.html>.

-- 
|       Michael Olson  |  FSF Associate Member #652     |
| http://mwolson.org/  |  Hobbies: Lisp, HCoop          |
| Projects: Emacs, Muse, ERC, EMMS, ErBot, DVC, Planner |
`-------------------------------------------------------'

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Emacs Bazaar repository

[Christian Faulhammer]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

Hi,

Jason Earl <jearl@xmission.com>:

> This repository is obviously something that should mostly interest
> Emacs developers, but I'm sending this email to the bazaar mailing
> list as well in case they are curious.

 And to throw in something else about speed: I tested the various
dVCS repositories available and compared them regarding taken time and
size of repository on disk.  This was the initial
checkout/clone/branch/whatever only, subsequent updates are faster for
all of them:

Bzr 1.1 and 1.3:
	40m9.579s
	397M

Git 1.5.4.4:
	8m31.483s
	128M

Arch/Tla 1.3.5:
	10m21.008s
	286M

V-Li

-- 
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode

<URL:http://www.faulhammer.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs Bazaar repository

[Thien-Thi Nguyen]

() Michael Olson <mwolson@gnu.org>
() Sun, 23 Mar 2008 23:19:00 -0700

   Here is a write-up explaining how I do all this:
   <http://mwolson.org/projects/GitCvsSync.html>.

Thanks.

I note that both "cvs export" and "git clone" accept a destination
directory argument; you can elide the "grab SRC; mv SRC DST" into
"grab SRC DST", like so:

cvs:  cvs checkout $repo -d cvs-emacs
git:  git clone    $repo    git-emacs

Not a big deal; the writeup is very clear, as it stands.

thi

Re: Emacs Bazaar repository

[Giorgos Keramidas]

On Sat, 15 Mar 2008 17:39:06 +0530, dhruva <dhruvakm@gmail.com> wrote:
> On Sat, Mar 15, 2008 at 3:56 AM, Martin Geisler <mg@daimi.au.dk> wrote:
>>  And even if you have several heads in your Mercurial repository, you can
>>  certainly still pull in new changesets.
>
> I have seen it and have used it too. The problem comes (from my
> experience) when you have to push. I agree you can create a new named
> branch and just pull in changes or do a forced pull which will create
> a new head. Suppose I want to keep multiple named branches active and
> yet push to a remote repository from one of the named branch, IMO it
> is not possible with mercurial. I would be happy to know if there is a
> way to do it.

Of course there is:

    hg push --force

This will inhibit the warning that you are `creating new heads', if you
push a new named branch.

Re: Emacs Bazaar repository

[David Kastrup]

Johannes Weiner <hannes@saeurebad.de> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> This question is over and decided.
>> We will use GNU Bzr, because it is a GNU package.
>
> Wow, that sounds almost democratic.

GNU never was a democratic project.  The masses are coming over to GNU,
not the other way round.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Emacs repository benchmark: bzr and git

[Jonathan Lange]

On Wed, Mar 19, 2008 at 1:53 PM, Richard Stallman <rms@gnu.org> wrote:
>      To the emacs maintainers and decision makers: What more information
>     is required to convince bzr is not the right tool at the present
>     moment?
>
>  This decision is not a decision for the present moment.  It is a long
>  term decision.
>
>  So it would be better to wait a few months while Bzr developers
>  improve it, than to make some other "temporary" decision that would
>  probably be hard to reverse.
>

I've tried to get a list of things that Bazaar needs to improve from
trawling through this list and bugs on the Bazaar tracker.

So far I have:
  * bzr log needs to be much faster for single files[1] and for
subsets of history.
  * bzr diff -r <something> needs to take at most a couple of seconds
  * bzr st <single file> needs to be instantaneous.
  * Must be able to get the diff between the current branch and its
parent very quickly.

These are all performance related. I can't recall coming across any
other blockers, but then I haven't been following every email on this
& related threads.

If I've missed anything, please let me know. If I've included anything
that's not really necessary, let me know. If you want, file bugs on
https://bugs.launchpad.net/bzr/+bugs and give them the tag 'emacs'.

jml

[1] Bazaar tends to assume you want to work with the whole tree for
every operation. I think this might be the root cause of a lot of the
disappointment in Bazaar's performance.

Re: Emacs repository benchmark: bzr and git

[Stephen J. Turnbull]

The bazaar people already know this, removing bazaar from the
addressees.

Jonathan Lange writes:

 > I've tried to get a list of things that Bazaar needs to improve from
 > trawling through this list and bugs on the Bazaar tracker.

Robert Collins posted an excellent summary of why the "pack" format
repo will make a foundation for rapid progress on performance, and
also summarized his view of what things need fixing.  (Sorry, I"m just
before getting on a plane, but it's in the "bazaar" thread on "more
plans, less excuses" or something like that.)

 > [1] Bazaar tends to assume you want to work with the whole tree for
 > every operation. I think this might be the root cause of a lot of the
 > disappointment in Bazaar's performance.

Do you mean whole tree, or whole history?  If it's "whole tree", then
git is the obvious counterexample, with most operations on whole trees
being O(1).  Even for "whole history", gitk is an order of magnitude
faster than bzr log.

Re: Emacs repository benchmark: bzr and git

[Stefan Monnier]

> I've tried to get a list of things that Bazaar needs to improve from
> trawling through this list and bugs on the Bazaar tracker.

> So far I have:
>   * bzr log needs to be much faster for single files[1] and for
> subsets of history.
>   * bzr diff -r <something> needs to take at most a couple of seconds
>   * bzr st <single file> needs to be instantaneous.
>   * Must be able to get the diff between the current branch and its
> parent very quickly.

Actually, this last point is the same as the second.  As for the
remaining three they come in this order:

   * bzr st <single file> needs to be instantaneous.
   * bzr diff -r <something> needs to take at most a couple of seconds
   * bzr log needs to be much faster for single files[1] and for
     subsets of history.

The first is very serious since it make vc-bzr unusable.  Also it's very
easy to fix by providing a new option that prevents outputting the list
of pending changes (which is the operation that takes a long time, and
we don't need that info anyway).

> These are all performance related. I can't recall coming across any
> other blockers, but then I haven't been following every email on this
> & related threads.

Also, given the current performance problems, we haven't tried
much more.  E.g. I have no idea how easy it will be to keep Bzr-Emacs in
sync with Gnus's repository.

> [1] Bazaar tends to assume you want to work with the whole tree for
> every operation.  I think this might be the root cause of a lot of the
> disappointment in Bazaar's performance.

Actually, as far as I can tell, most of the above problems are unrelated
to "single file vs whole tree": it takes about the same time to do it on
a single file than on the whole tree.


        Stefan

Re: Emacs repository benchmark: bzr and git

[dhruva]

Hi,

On Thu, Mar 27, 2008 at 8:25 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>  Also, given the current performance problems, we haven't tried
>  much more.  E.g. I have no idea how easy it will be to keep Bzr-Emacs in
>  sync with Gnus's repository.

I have tried keeping my bzr repo of Emacs updated. It is extremely
slow compared to GIT. When a connection breaks during a 'bzr pull', I
need to restart the slow process all over again. I would do the same
with GIT too, since the performance is good, I have nothing to
complain. I would still prefer if the tools could wait for sometime
before giving up. Usually when I use WIFI connectivity, there are
intermittent connection breakage due to feeble signal, I would be
happy if the tools (SCM) could just retry the previous request a few
times before giving up.

-dhruva

Re: Emacs repository benchmark: bzr and git

[Vincent Ladeuil]

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

    >> I've tried to get a list of things that Bazaar needs to improve from
    >> trawling through this list and bugs on the Bazaar tracker.

    >> So far I have:
    >> * bzr log needs to be much faster for single files[1] and for
    >> subsets of history.
    >> * bzr diff -r <something> needs to take at most a couple of seconds
    >> * bzr st <single file> needs to be instantaneous.
    >> * Must be able to get the diff between the current branch and its
    >> parent very quickly.

    Stefan> Actually, this last point is the same as the second.  As for the
    Stefan> remaining three they come in this order:

    Stefan>    * bzr st <single file> needs to be instantaneous.
    Stefan>    * bzr diff -r <something> needs to take at most a couple of seconds
    Stefan>    * bzr log needs to be much faster for single files[1] and for
    Stefan>      subsets of history.

    Stefan> The first is very serious since it make vc-bzr
    Stefan> unusable.

I know that one since it's the precise reason why I stopped using
vc-bzr months ago.

Can you tell us which precise information is needed by vc-bzr (my
understanding, dating from vc-cvs years ago now, is that it wants
to display the file version in the mode line).

I suspect that 'bzr st' aim does not fit vc-bzr needs at all
(even if it's the closest available command from bzr).

    Stefan> Also it's very easy to fix by providing a new option
    Stefan> that prevents outputting the list of pending changes
    Stefan> (which is the operation that takes a long time, and
    Stefan> we don't need that info anyway).

Correct. But with a better understanding of the needs we may find
a better solution without special-casing bzr st.

<snip/>

    Stefan> Also, given the current performance problems, we
    Stefan> haven't tried much more.  E.g. I have no idea how
    Stefan> easy it will be to keep Bzr-Emacs in sync with Gnus's
    Stefan> repository.

Does that mean that Gnus is maintained as a separate project and
regularly kept in sync/imported in the emacs CVS repository ?

If yes, the best solution will be what bzr call 'nested trees'
which is currently working on.

    >> [1] Bazaar tends to assume you want to work with the whole
    >> tree for every operation.  I think this might be the root
    >> cause of a lot of the disappointment in Bazaar's
    >> performance.

    Stefan> Actually, as far as I can tell, most of the above
    Stefan> problems are unrelated to "single file vs whole
    Stefan> tree": it takes about the same time to do it on a
    Stefan> single file than on the whole tree.

I think you both agree :) Internally bzr tends (this is less and
less true) to process the whole tree and then filter the results.

     Vincent

Re: Emacs Bazaar repository

[Johannes Weiner]

Hi,

David Kastrup <dak@gnu.org> writes:

> Johannes Weiner <hannes@saeurebad.de> writes:
>
>> Richard Stallman <rms@gnu.org> writes:
>>
>>> This question is over and decided.
>>> We will use GNU Bzr, because it is a GNU package.
>>
>> Wow, that sounds almost democratic.
>
> GNU never was a democratic project.  The masses are coming over to GNU,
> not the other way round.

Thanks for clarifying.  I also misunderstood the context of Richard's
statement, sorry.

	Hannes

Re: Emacs repository benchmark: bzr and git

[Xavier Maillard]

   > [1] Bazaar tends to assume you want to work with the whole tree for
   > every operation.  I think this might be the root cause of a lot of the
   > disappointment in Bazaar's performance.

   Actually, as far as I can tell, most of the above problems are unrelated
   to "single file vs whole tree": it takes about the same time to do it on
   a single file than on the whole tree.

What's more, it takes really long time to perform things even
with a pretty new bazaar repository. I have switched a few
projects just to take measure and no matter how big is the
history, all operations are rather expensive in time compared to
any other DVC I am using.

As my python skills are rather small, I can't contribute much
to help "profiling" the code.

	Xavier

Re: Emacs Bazaar repository

[Xavier Maillard]

   On Thu, Mar 13, 2008 at 3:06 PM, Karl Fogel <kfogel@red-bean.com> wrote:
   >    2) I run Debian GNU/Linux ("testing" dist) and have installed the
   >       'bzr' package now using 'apt-get'.  Given the many variants of
   >       Bazaar(s) that have existed over the years, I can't be positive
   >       that 'bzr' is the right package; if it is not, someone please post
   >       here saying what is :-).
   >

   'bzr' is the right one. Work is underway to have the package renamed
   to 'bazaar'.

Why do you want to rename it ?

	Xavier

Re: Emacs Bazaar repository

[Xavier Maillard]

   My first impression is that bzr is slow, so slow that it is completely
   unusable.  How can it come that a simple bzr log takes more than a
   minute to even start?  Even cvs log is instantaneous in comparison,
   although it has to request the log from the server.

This slowness tenders to make any emacs oriented tool (vc-bzr,
dvc, ...) almost useless and boring. There is something even
worst: slowness makes is a major problem for the "commit often"
slogan. Who want to spend/loose roughly 30s to even start to do
anything ?

I have switched a few projects a few days ago and the experiment
is really painful and certainly not entertaining.

I will persevere on using bzr but dunno how long :)

Although the performances are quite bad, the ease of use of the
whole project is interesting.

	Xavier

Re: Emacs Bazaar repository

[Xavier Maillard]

   Karl Fogel <kfogel@red-bean.com> writes:

   > All of the DVCSs seem good.  No one marshalled any compelling arguments
   > in favor of one versus the other on technical grounds, and all other
   > things being equal, RMS (and maybe some others, perhaps including Yidong
   > and Stefan?) preferred bzr because it is a GNU project.

   In its current form bzr is not usable for a project of the size of
   Emacs.

Although performances are pretty bad in the current state, things
can be improved if we discuss with the bazaar team. The latest
pull from bzr are giving me some hope for the future -i.e. bzr
log --line/--short is pretty quick now (even with the emacs bzr
repository).

We should try to help improving bazaar as most as possible.

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Emacs repository benchmark: bzr and git

[Xavier Maillard]

   I personally do not see a single mail which can tilt the scale
   in favor of bzr except it is part of GNU project.

The "except it is part of GNU project" is the _main_ reason.
Agreed or not, that's not the question. Maybe there is a need to
explain on gnu.org, why a GNU project *should* support another
GNU project when there are many free (or non-free) competitor
projects.

	Xavier

Re: Emacs Bazaar repository

[Xavier Maillard]

    I just hope it the decision is based on technical facts rather than
   affiliations and emotions...

I hope not. Technical facts are not, from my point of view, a
sufficient condition. As stated by RMS, and even if I did not
agree at first, we must support a GNU project when possible.

We can help in making bzr feets our needs.

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Emacs Bazaar repository

[Xavier Maillard]

   Johannes Weiner <hannes@saeurebad.de> writes:

   > Richard Stallman <rms@gnu.org> writes:
   >
   >> This question is over and decided.
   >> We will use GNU Bzr, because it is a GNU package.
   >
   > Wow, that sounds almost democratic.

   GNU never was a democratic project.  The masses are coming over to GNU,
   not the other way round.

Well spoken ! That is the free aspect of the software that
permits to reach quality and a good technical level in my
opinion. In conclusion, we can help bazaar reach an acceptable
usable state (for emacs) because it's free software.

It is not currently the case but it will.

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Emacs repository benchmark: bzr and git

[Stefan Monnier]

Stefan> Actually, this last point is the same as the second.  As for the
Stefan> remaining three they come in this order:

Stefan> * bzr st <single file> needs to be instantaneous.
Stefan> * bzr diff -r <something> needs to take at most a couple of seconds
Stefan> * bzr log needs to be much faster for single files[1] and for
Stefan> subsets of history.

Stefan> The first is very serious since it make vc-bzr
Stefan> unusable.

Actually, I've just installed a patch in Emacs's trunk to make vc-bzr.el
parse the dirstate file more completely, so `bzr status' is not needed
nearly as often.  But I don't have any documentation about the dirstate
format, so it's not very satisfactory.

Anybody knows where I can find some doc about that file's format?

> Can you tell us which precise information is needed by vc-bzr (my
> understanding, dating from vc-cvs years ago now, is that it wants
> to display the file version in the mode line).

It wants to know if the file is under Bzr's control, whether it's
added/removed/edited/uptodate and what is the current revision number
(basically).  Ideally also if there are conflicts (tho it currently
doesn't use `bzr status' for that but just looks for the .MINE and
friends instead).

> I suspect that 'bzr st' aim does not fit vc-bzr needs at all
> (even if it's the closest available command from bzr).

It seems to fit just fine, except it takes an insance amount of time to
give me the list of pending merges.  2 problems with that:
1 - performance
2 - the list includes merges that are not pertinent (because that merge
    does not affect the specified file(s)).
Admittedly, the worst part

Stefan> Also, given the current performance problems, we
Stefan> haven't tried much more.  E.g. I have no idea how
Stefan> easy it will be to keep Bzr-Emacs in sync with Gnus's
Stefan> repository.

> Does that mean that Gnus is maintained as a separate project and
> regularly kept in sync/imported in the emacs CVS repository ?

Yes, the two branches are sync'd both ways (I believe they both
currently use CVS as the main repository and the sync'ing is done
semi-automatically via Arch).

> If yes, the best solution will be what bzr call 'nested trees'
> which is currently working on.

I'm not sure this will be sufficient: the Gnus files are not all neatly
confined within a single directory.  Arch handles such cases just fine
(patches to Emacs files that aren't in Gnus are just ignored and the
rename support is sufficiently good to be able to figure out which
files correspond to which between the two projects; better yet, the two
projects were brought together after the fact and the renames are
properly handled even without a common ancestor thanks to the use of
arch-tags in the files).

> I think you both agree :) Internally bzr tends (this is less and
> less true) to process the whole tree and then filter the results.

Could be.  I hope not: the lessons learned from Arch should include
avoiding to touch/construct/look-at files unless absolutely necessary.
But I'm sure it's not the only problem: the speed difference between
"bzr tags" and "bzr tags --show-ids" can't be due to
whole-tree operations.


        Stefan

Re: Emacs repository benchmark: bzr and git

[James Westby]

On Fri, 2008-03-28 at 21:17 -0400, Stefan Monnier wrote:
> Actually, I've just installed a patch in Emacs's trunk to make vc-bzr.el
> parse the dirstate file more completely, so `bzr status' is not needed
> nearly as often.  But I don't have any documentation about the dirstate
> format, so it's not very satisfactory.
> 
> Anybody knows where I can find some doc about that file's format?
> 

bzrlib/dirstate.py

No, I'm not being facetious, there's a BNF at the top of the file
that should help you. 

Thanks,

James

Re: Emacs repository benchmark: bzr and git

[Stefan Monnier]

>> Anybody knows where I can find some doc about that file's format?

> bzrlib/dirstate.py

Thanks, exactly what I was looking for, although the "current tree vs
second tree" distinction is very much unclear to me.


        Stefan

Re: Emacs repository benchmark: bzr and git

[James Westby]

On Fri, 2008-03-28 at 23:09 -0400, Stefan Monnier wrote:
> >> Anybody knows where I can find some doc about that file's format?
> 
> > bzrlib/dirstate.py
> 
> Thanks, exactly what I was looking for, although the "current tree vs
> second tree" distinction is very much unclear to me.

I *think* it's actually one or more trees. The first tree is the
left hand parent. Any other tress are the state in the other
parents, i.e. merged revisions.

Thanks,

James

Re: Emacs repository benchmark: bzr and git

[David Cournapeau]

Xavier Maillard wrote:
> What's more, it takes really long time to perform things even
> with a pretty new bazaar repository. I have switched a few
> projects just to take measure and no matter how big is the
> history, all operations are rather expensive in time compared to
> any other DVC I am using.
>
>   

That's strange. Are you comparing to mercurial or git ? Which version of 
bzr were you using (bzr --version) ?

> As my python skills are rather small, I can't contribute much
> to help "profiling" the code.
>   

I think giving the name of the projects (if they are open source) and 
the commands you were using would be enough to see an eventual problem. 
I don't have experience for large projects with long history (the 
biggest project I use bzr with is ~ 2000 files with ~ 4000 revisions), 
but I compared those with mercurial a few months ago, and they were 
comparable in speed for branch/st/ci/diff commands (log and blame were 
much slower, but again, if there is no history, I don't think that's 
what you are talking about).

cheers,

David

Re: Emacs repository benchmark: bzr and git

[Stefan Monnier]

> I *think* it's actually one or more trees. The first tree is the
> left hand parent. Any other tress are the state in the other
> parents, i.e. merged revisions.

I don't think so.  The "second tree" seems to correspond to the
leftmost parent.  The first tree seems to correspond to something else.
It's usually the same as the second, but when I did a merge, the first
tree seems to describe something like the state of tree at the end of
the merge command, and it dosn't have sha1 for the files with conflicts
(and maybe for others as well).

When merging, there are more than 2 trees (typically just 3: the
"current", the "second" (i.e. leftmost parent), and some other which
I guess corresponds to the other parent).


        Stefan

Re: Emacs repository benchmark: bzr and git

[Andreas Schwab]

Xavier Maillard <xma@gnu.org> writes:

>    I personally do not see a single mail which can tilt the scale
>    in favor of bzr except it is part of GNU project.
>
> The "except it is part of GNU project" is the _main_ reason.

When an arbitrary political decision wipes away all technical arguments
it is not helping free software in any way.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs repository benchmark: bzr and git

[David Kastrup]

Andreas Schwab <schwab@suse.de> writes:

> Xavier Maillard <xma@gnu.org> writes:
>
>>    I personally do not see a single mail which can tilt the scale
>>    in favor of bzr except it is part of GNU project.
>>
>> The "except it is part of GNU project" is the _main_ reason.
>
> When an arbitrary political decision wipes away all technical arguments
> it is not helping free software in any way.

Uh, an arbitrary political decision wiping away all technical arguments
is what started free software in the first place.

The question is not whether the decision is political or technical.  The
free software movement is, after all, a political one.  The question is
whether it is the right political decision.  I am not convinced of that.

But your argument is putting the cart before the horse.  We should let
ourselves be directed by our goals, not by letting gravity take its
course at the place where we already are.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Emacs repository benchmark: bzr and git

[dhruva]

I personally find it odd to compare between a GNU product and another
GPL software. IMHO, we should be treating all GPLed softwares equally.
Hence, comparison between git and bzr can be termed as apple to apple
comparision.

-dhruva

On 3/29/08, David Kastrup <dak@gnu.org> wrote:
> Andreas Schwab <schwab@suse.de> writes:
>
> > Xavier Maillard <xma@gnu.org> writes:
> >
> >>    I personally do not see a single mail which can tilt the scale
> >>    in favor of bzr except it is part of GNU project.
> >>
> >> The "except it is part of GNU project" is the _main_ reason.
> >
> > When an arbitrary political decision wipes away all technical arguments
> > it is not helping free software in any way.
>
> Uh, an arbitrary political decision wiping away all technical arguments
> is what started free software in the first place.
>
> The question is not whether the decision is political or technical.  The
> free software movement is, after all, a political one.  The question is
> whether it is the right political decision.  I am not convinced of that.
>
> But your argument is putting the cart before the horse.  We should let
> ourselves be directed by our goals, not by letting gravity take its
> course at the place where we already are.
>
> --
> David Kastrup, Kriemhildstr. 15, 44793 Bochum
>
>
>

Re: Emacs repository benchmark: bzr and git

[David Kastrup]

dhruva <dhruvakm@gmail.com> writes:

> I personally find it odd to compare between a GNU product and another
> GPL software. IMHO, we should be treating all GPLed softwares equally.
> Hence, comparison between git and bzr can be termed as apple to apple
> comparision.

Not quite.  git is licensed as "GPL v2 only" which means that its code
can't be interchanged freely with code of the GNU project.  While it is
free software in itself, this puts it at a different level with regard
to how much interchange is possible.

So we are not talking apples and apples here.  As I said, I am not sure
that the decision is the best decision even if viewed from a political
angle.  But it is not as arbitrary as some want to paint it.

Re: Emacs repository benchmark: bzr and git

[Randal L. Schwartz]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

David> Uh, an arbitrary political decision wiping away all technical arguments
David> is what started free software in the first place.

Only for the revisionists.  Free software existed long before the GNU Project.

The GNU Project's contribution was a legally enforcable license to ensure
share-and-share-alike for authors who chose to use the legal system to enforce
their politics.  This was definitely an important contribution at the time,
but it wasn't the thing that "started free software".

Those of us who have been around from the beginning must tell the story *as*
it happened, not replay the revisionist's story.  In other words, if you don't
lie, I won't have to respond.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: Emacs repository benchmark: bzr and git

[Óscar Fuentes]

merlyn@stonehenge.com (Randal L. Schwartz) writes:

> David> Uh, an arbitrary political decision wiping away all technical arguments
> David> is what started free software in the first place.
>
> Only for the revisionists.  Free software existed long before the GNU Project.
>
> The GNU Project's contribution was a legally enforcable license to ensure
> share-and-share-alike for authors who chose to use the legal system to enforce
> their politics.  This was definitely an important contribution at the time,
> but it wasn't the thing that "started free software".

Free Software, ("Free" in the GNU sense), is nothing without a legally
enforceable license that protects its freedom. Maybe you are talking
about what later was known as Open Source? (BSD license, etc).

[snip]

-- 
Oscar

Re: Emacs repository benchmark: bzr and git

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Mar 29, 2008 at 05:35:36PM +0100, Óscar Fuentes wrote:
> merlyn@stonehenge.com (Randal L. Schwartz) writes:
> 
> > David> Uh, an arbitrary political decision wiping away all technical arguments
> > David> is what started free software in the first place.
> >
> > Only for the revisionists.  Free software existed long before the GNU Project.
> >
> > The GNU Project's contribution was a legally enforcable license to ensure
> > share-and-share-alike for authors who chose to use the legal system to enforce
> > their politics.  This was definitely an important contribution at the time,
> > but it wasn't the thing that "started free software".
> 
> Free Software, ("Free" in the GNU sense), is nothing without a legally
> enforceable license that protects its freedom. Maybe you are talking
> about what later was known as Open Source? (BSD license, etc).

Oh, noes. You are confused. BSD *is* free by FSF standards[1]. Open
Source is a term coined much later for those who want the advantages of
free without the politics (or put in another terms: to avoid scaring
the suits).

Watch Theo de Raadts rants about _even GNU_ being not free enough to
know what I mean :-)

Open == Free - ethics.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFH7obDBcgs9XrR2kYRAvH5AJ9rrq3RqHQi0OaOr6T5v3QG5NdSYwCfcBCK
EMV2f0HNAZIIVSMSjHNvU0Q=
=kk2O
-----END PGP SIGNATURE-----

Re: Emacs repository benchmark: bzr and git

[Stephen J. Turnbull]

Randall Schwartz writes:

 > > The GNU Project's contribution was a legally enforcable license
 > > to ensure share-and-share-alike for authors who chose to use the
 > > legal system to enforce their politics.  This was definitely an
 > > important contribution at the time, but it wasn't the thing that
 > > "started free software".

I basically agree, but one quibble.  My understanding is that a
second, equally important, contribution was the very idea of a free
software distribution (ie, public release of a large suite of free
software, preferably a full operating system).  I'm fairly sure that
the BSD NET/1 release, and the proposal to greatly expand the
distribution as NET/2, was triggered by hearing about GNU.  Sort of a
"hey, that's it!  That's what we've wanted to do all along!"

Óscar Fuentes replies:

 > Free Software, ("Free" in the GNU sense), is nothing without a
 > legally enforceable license that protects its freedom. Maybe you
 > are talking about what later was known as Open Source? (BSD
 > license, etc).

Those *are* legally enforceable licenses that protect the licensed
software's freedom.  Every line of the covered work is fully free, in
perpetuity, as long as a single copy under that license is publicly
available.

Second, please be careful to distinguish between "free software" as
software (which is what Randall was talking about, and is essentially
identical to open source software as defined in the Open Source
Definition), and the "Free Software Movement", which is a political
movement.  Any software licensed under a free software license is free
software, regardless of the political opinions of the licensor.

BTW, the Open Source movement is quite hospitable to people whose
primary goal is liberty (eg, the "Quaker faction").  You just have to
believe in liberty enough that you can cooperate happily with the
"economists" who see liberty as instrumental to productivity, rather
than the other way around.

Re: Emacs repository benchmark: bzr and git

[Richard Stallman]

    When an arbitrary political decision wipes away all technical arguments
    it is not helping free software in any way.

The rule that GNU packages should support each other
helps make the GNU system as a whole work better.

Re: Emacs repository benchmark: bzr and git

[Richard Stallman]

Free software existed before we started GNU, but was few and
scattered; the only way you could use it was as add-ons to a non-free
system.  The GNU system made it possible to use a computer and have
freedom.

    The GNU Project's contribution was a legally enforcable license to ensure
    share-and-share-alike for authors who chose to use the legal system to enforce
    their politics.

The GNU Project developed a free operating system, GNU.  (See
http://www.gnu.org/gnu/the-gnu-project.html.)  To do this, we
developed many programs, which are GNU packages.  We still develop
more GNU packages and also accept existing programs into the GNU
Project when their developers wish.

As a supporting activity, we also developed a license for releasing
these programs as free software and defend freedom for all users.
That is the GNU GPL.

Re: Emacs repository benchmark: bzr and git

[Jonathan Rockway]

* On Sun, Mar 30 2008, Richard Stallman wrote:
>     When an arbitrary political decision wipes away all technical arguments
>     it is not helping free software in any way.
>
> The rule that GNU packages should support each other
> helps make the GNU system as a whole work better.

Why can't we just make git part of the GNU system?  Is it a copyright
assignment thing?

-- 
print just => another => perl => hacker => if $,=$"

Re: Emacs repository benchmark: bzr and git

[Richard Stallman]

    Why can't we just make git part of the GNU system?

We could include it in the GNU system, but its developers are not
likely to want to make it a GNU package.

Re: Emacs repository benchmark: bzr and git

[Xavier Maillard]

   Xavier Maillard wrote:
   > What's more, it takes really long time to perform things even
   > with a pretty new bazaar repository. I have switched a few
   > projects just to take measure and no matter how big is the
   > history, all operations are rather expensive in time compared to
   > any other DVC I am using.

   That's strange. Are you comparing to mercurial or git ? Which version of 
   bzr were you using (bzr --version) ?

I am using bzr.dev.

   > As my python skills are rather small, I can't contribute much
   > to help "profiling" the code.
   >   

   I think giving the name of the projects (if they are open source) and 
   the commands you were using would be enough to see an eventual problem. 
   I don't have experience for large projects with long history (the 
   biggest project I use bzr with is ~ 2000 files with ~ 4000 revisions), 
   but I compared those with mercurial a few months ago, and they were 
   comparable in speed for branch/st/ci/diff commands (log and blame were 
   much slower, but again, if there is no history, I don't think that's 
   what you are talking about).

The major bottleneck happens with GNU Emacs repository. My
projetcs are pretty small (in files and revisions) but they
"lag".

	Xavier

Re: Emacs repository benchmark: bzr and git

[Xavier Maillard]

   Xavier Maillard <xma@gnu.org> writes:

   >    I personally do not see a single mail which can tilt the scale
   >    in favor of bzr except it is part of GNU project.
   >
   > The "except it is part of GNU project" is the _main_ reason.

   When an arbitrary political decision wipes away all technical arguments
   it is not helping free software in any way.

Hopefully, a genius (or visionary) took a political/ethical
decision twenty years ago. Today, we have the GNU which is
technically and ethically superior to any other system available
around.

Souvenirs, souvenirs !


	Xavier

Re: Emacs repository benchmark: bzr and git

[Xavier Maillard]

   I personally find it odd to compare between a GNU product and another
   GPL software. IMHO, we should be treating all GPLed softwares equally.
   Hence, comparison between git and bzr can be termed as apple to apple
   comparision.

Who will support GNU projects if even between GNU teams, we do
not support/promote them ? It's all about what RMS calls
"fairness".

We still can compare GNU bazaar with git, mercurial and al but we
_should_ concentrate on promoting GNU bazaar as part of the GNU.

I am a long time git user and will probably continue to use it
but I also want GNU bazaar to improve. So I trying to join and to
help in reaching this goal.

	Xavier