Bzr switch

Bzr switch

[Stefan Monnier]

Any news on this front?


        Stefan

Re: Bzr switch

[Karl Fogel]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> Any news on this front?

Bazaar 1.16 was released on on the 18th.  It has support for the "2a"
format that we want to host Emacs in (the format known during
development as "brisbane-core", and now called "2a").  That format will
be the default in Bazaar 2.0, though it is not the default in 1.16.x.

1.16 was followed by 1.16.1 a few days later, which fixed a couple of
bugs (see [1]).

1.16.1 is currently getting a fair amount of real-world testing on
another large source tree (see [2]).  I think it would be wise to let us
test it there for another week or two, and then proceed with the Emacs
switchover.  1.16.1 has been pretty stable so far, but it's still a new
release.  Let's let the water warm up a bit, then we jump in.

-Karl

[1] https://lists.ubuntu.com/archives/bazaar/2009q2/060008.html

[2] https://lists.launchpad.net/launchpad-users/msg05078.html

Re: Bzr switch

[Daniel Clemente]

El dom, jun 28 2009 a les 08:53, Karl Fogel va escriure:
>
> Bazaar 1.16 was released on on the 18th.  It has support for the "2a"
> format that we want to host Emacs in (the format known during
> development as "brisbane-core", and now called "2a").  That format will
> be the default in Bazaar 2.0, though it is not the default in 1.16.x.
>

  What will happen when older clients try to connect?
  Even if they are asked to upgrade to 1.16, this may be a good thing. (Upgrading is as easy as: bzr branch lp:bzr; cd bzr; ./bzr).

>
> 1.16.1 is currently getting a fair amount of real-world testing on
> another large source tree (see [2]).  I think it would be wise to let us
> test it there for another week or two, and then proceed with the Emacs
> switchover.  1.16.1 has been pretty stable so far, but it's still a new
> release.  Let's let the water warm up a bit, then we jump in.

  Nice to see some dates.

  Some questions from the links I found in [1] :
- The Savannah ticket about this is still open: [2] Is Savannah officially ready?
- There seem to be still open Bazaar bugs/improvements with the tag emacs-adoption: [3]. Are they blocking?

  I saw that some Bazaar bugs which affected Emacs were solved due to your work; thanks.

[1] http://www.emacswiki.org/emacs/EmacsBzrSwitchover
[2] http://savannah.gnu.org/support/index.php?106612
[3] https://bugs.edge.launchpad.net/bzr/+bugs?field.searchtext=&orderby=-importance&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.has_cve.used=&field.tag=emacs-adoption&field.tags_combinator=ANY&search=Search

-- Daniel

Re: Bzr switch

[Daniel Clemente]

  By the way, the test repository given in [1] doesn't seem to be a branch:

$ bzr branch bzr://bzr.notengoamigos.org/emacs-merges-ce/master/
bzr: ERROR: Not a branch: "bzr://bzr.notengoamigos.org/emacs-merges-ce/master/".

[1]: http://www.emacswiki.org/emacs/EmacsBzrSwitchover

-- Daniel

Re: Bzr switch

[Stefan Monnier]

>> Any news on this front?
> Bazaar 1.16 was released on on the 18th.  It has support for the "2a"
> format that we want to host Emacs in (the format known during
> development as "brisbane-core", and now called "2a").  That format will

Actually, I don't have a preference for that format.  If I had
a preference it'd be for the 1.9 format.

But before we can switch over we still need a test repository set up on
Savannh so we can make sure that Savannah is working correctly
(including loggerhead, the commit mailing-list, ...).


        Stefan

Re: Bzr switch

[Karl Fogel]

Daniel Clemente <dcl441-bugs@yahoo.com> writes:
>   What will happen when older clients try to connect?
>   Even if they are asked to upgrade to 1.16, this may be a good
>   thing. (Upgrading is as easy as: bzr branch lp:bzr; cd bzr; ./bzr).

I don't know if they get a nice error telling them to upgrade, or a
not-so-nice error.  I suspect the former, but haven't tested it yet.
The solution is the same either way: upgrade to 1.16.1 or higher.

For our purposes, we should just publish that pulling Emacs requires
Bazaar >= 1.16.1 and tell people to upgrade if necessary.  1.17 will be
out very soon, so that will probably be what we recommend, actually.

(Jason and Andreas, if you want to make a new testing branch now using
the "--2a" format, that would be great...)

>   Nice to see some dates.
>
>   Some questions from the links I found in [1] :
> - The Savannah ticket about this is still open: [2] Is Savannah
>   officially ready?

I think Savannah will need to upgrade Bazaar to get loggerhead
(web-viewing) support for the new branch format.

> - There seem to be still open Bazaar bugs/improvements with the tag
> emacs-adoption: [3]. Are they blocking?

None of those look like blockers to me.

> I saw that some Bazaar bugs which affected Emacs were solved due to
> your work; thanks.

Well, I can't take credit for the coding (except for one bug), but on
behalf of the real Bazaar developers: you're welcome! :-)

Stefan Monnier writes:
> Actually, I don't have a preference for that format.  If I had
> a preference it'd be for the 1.9 format.

I strongly recommend the new format.  It is faster and smaller, in ways
that will make a difference for a large, deep-history project like Emacs.
In particular, 'log -v' times are faster, though I wish they were faster
still.

> But before we can switch over we still need a test repository set up on
> Savannh so we can make sure that Savannah is working correctly
> (including loggerhead, the commit mailing-list, ...).

Yes.  Again, I'd like to just give Bazaar 1.16.1 a couple of weeks more
testing -- I don't want to ask the Savannah admins to upgrade only to
have to ask them to do it again shortly afterwards.  I'm keeping an eye
on our Bazaar testing (for the Launchpad.net open-sourcing), and will
come back and ping the Savannah admins very soon.

No objection to starting testing earlier, of course!  Just in terms of
my own schedule, and a desire to avoid doing work twice, I'm planning to
wait a couple of weeks before making any noises at Savannah.  If anyone
wants to take this and run with it sooner, though, I'm all in favor.

However, we can start testing a branch by itself before then.  (See
above note to Jason and Andreas.)

-Karl

The references from Daniel Clemente's mail:
> [1] http://www.emacswiki.org/emacs/EmacsBzrSwitchover
> [2] http://savannah.gnu.org/support/index.php?106612
> [3] https://bugs.edge.launchpad.net/bzr/+bugs?field.searchtext=&orderby=-importance&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.has_cve.used=&field.tag=emacs-adoption&field.tags_combinator=ANY&search=Search

Re: Bzr switch

[Stefan Monnier]

>> Actually, I don't have a preference for that format.  If I had
>> a preference it'd be for the 1.9 format.
> I strongly recommend the new format.  It is faster and smaller, in ways
> that will make a difference for a large, deep-history project like Emacs.
> In particular, 'log -v' times are faster, though I wish they were faster
> still.

I don't care much about such incremental improvements.  Factors of 2 are
nice but don't change my point of view: having to manually install Bzr
just to access Emacs's repository is a bad idea.
Especially given that the format can be changed in the future, so
there's no reason to hurry up adoption of the new format.

> Yes.  Again, I'd like to just give Bazaar 1.16.1 a couple of weeks more
> testing -- I don't want to ask the Savannah admins to upgrade only to
> have to ask them to do it again shortly afterwards.  I'm keeping an eye

In the Bzr world, there's always something new coming up providing this
and that performance improvement.  So there's no point waiting for the
next one, cause when the time is passed, you'll end up wanting to wait
for the next next one, etc... ad nauseam.

> No objection to starting testing earlier, of course!  Just in terms of
> my own schedule, and a desire to avoid doing work twice, I'm planning to
> wait a couple of weeks before making any noises at Savannah.  If anyone
> wants to take this and run with it sooner, though, I'm all in favor.

I'd much rather work on getting the 1.9 format working on Savannah right
now, so we can switch to Bzr ASAP.
Of course, Jason tells me that the conversion is currently broken for
non-bleeding-edge formats, so maybe the 1.9 format is not really an
option.


        Stefan

Re: Bzr switch

[Karl Fogel]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> In the Bzr world, there's always something new coming up providing this
> and that performance improvement.  So there's no point waiting for the
> next one, cause when the time is passed, you'll end up wanting to wait
> for the next next one, etc... ad nauseam.

I understand what you mean, but This Time Is Different (or so I am told,
in no uncertain terms, by every Bazaar developer I talk to).

> Of course, Jason tells me that the conversion is currently broken for
> non-bleeding-edge formats, so maybe the 1.9 format is not really an
> option.

That may decide for us! :-)

Re: Bzr switch

[Stephen J. Turnbull]

Karl Fogel writes:
 > Stefan Monnier <monnier@iro.umontreal.ca> writes:
 > > In the Bzr world, there's always something new coming up providing this
 > > and that performance improvement.  So there's no point waiting for the
 > > next one, cause when the time is passed, you'll end up wanting to wait
 > > for the next next one, etc... ad nauseam.
 > 
 > I understand what you mean, but This Time Is Different (or so I am told,
 > in no uncertain terms, by every Bazaar developer I talk to).

Who are you talking to?  The list traffic tells a very different
story.  Mark Shuttleworth (Da Beeg Boos) wrote "Thou Shalt Have But
One Bazaar 2 Format", and got *lots* of pushback.  AFAIK he gave up;
the Bazaar devs surely act like that pronunciamento is "inoperative".
Robert Collins is *not* on board on "2a", specifically, looms are not
included in 2a, which means at least two variants of 2a will be out
there in the wild for a while.  (Silver lining: looms are usable as a
local improvement, you don't actually need them on the server for many
common workflows.)  Aaron Bentley is *not* on board on "2a",
specifically nested trees have not landed, but they will.  (Emacs
doesn't care, I think, but many projects want nested trees very badly.
It will land, not before 2.0, I suspect, but not too long thereafter.
This will mean a server upgrade, the formats are mutually incompatible
currently AIUI.)  Four variants.  And counting---don't kid yourself,
there will be more.

IMO (and that of the Python devs, see PEPs 374 and 385), Stefan is
right.  You need to settle on a version that's either Available Now or
Coming Soon To A GNU/Linux Distro Near You.  People who want to bleed
can always do that (to) themselves. :-)

Re: Bzr switch

[Miles Bader]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
>  > > In the Bzr world, there's always something new coming up providing this
>  > > and that performance improvement.  So there's no point waiting for the
>  > > next one, cause when the time is passed, you'll end up wanting to wait
>  > > for the next next one, etc... ad nauseam.
>  > 
>  > I understand what you mean, but This Time Is Different (or so I am told,
>  > in no uncertain terms, by every Bazaar developer I talk to).
>
> Who are you talking to?  The list traffic tells a very different story.

Karl's reply had such a cynical tone that I wasn't really sure whether he
was agreeing or disagreeing with Stefan!

-Miles

-- 
Dictionary, n.  A malevolent literary device for cramping the growth of
a language and making it hard and inelastic. This dictionary, however,
is a most useful work.

Re: Bzr switch

[Stephen J. Turnbull]

Miles Bader writes:
 > "Stephen J. Turnbull" <stephen@xemacs.org> writes:
 > >  > > In the Bzr world, there's always something new coming up providing this
 > >  > > and that performance improvement.  So there's no point waiting for the
 > >  > > next one, cause when the time is passed, you'll end up wanting to wait
 > >  > > for the next next one, etc... ad nauseam.
 > >  > 
 > >  > I understand what you mean, but This Time Is Different (or so I am told,
 > >  > in no uncertain terms, by every Bazaar developer I talk to).
 > >
 > > Who are you talking to?  The list traffic tells a very different story.
 > 
 > Karl's reply had such a cynical tone that I wasn't really sure whether he
 > was agreeing or disagreeing with Stefan!

Now that I reread it as I quoted him, I see your point!

Unfortunately, it really is hard to tell given that both Mark and the
users want format stability badly, and the developers *do* keep
promising that This Time Is Different (specifically, that from now on
they'll do a better job of keeping format changes out of the users'
faces, and that they're over the performance hump).  I think both of
those are actually true -- but neither promise is going to be kept
100%, and (on historical form) the bzr.devs are not going to be able
to resist the temptation to achieve performance improvements with
format tweaks.

Re: Bzr switch

[Robert Collins]

[-- Attachment #1: Type: text/plain, Size: 5881 bytes --]

Sorry for breaking the thread; I'm not on this list, and my mail client
doesn't seem to have a 'lie about history' field.

> From: Stephen J. Turnbull 
> Date: 2009-06-29 23:37:53 GMT
> 
> Karl Fogel writes:
> > Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> > > In the Bzr world, there's always something new coming up providing this
> > > and that performance improvement.  So there's no point waiting for the
> > > next one, cause when the time is passed, you'll end up wanting to wait
> > > for the next next one, etc... ad nauseam.
> > 
> > I understand what you mean, but This Time Is Different (or so I am told,
> > in no uncertain terms, by every Bazaar developer I talk to).

I think it is different. One thing that 2a gives us is *breathing space* -
until now we've been playing catchup - everytime we improved performance,
another group of users would pop up and say 'we want to use it, but damn its
slow'. We're not finished - thats true, be we are now definitively over the
performance hump. We haven't really promised to keep format changes out of
the way in the past - and we've had a lot of pressure to deploy every single
incremental improvement we made. We haven't changed the default format since
Octobert 2007, and it's my hope that the performance wins in 2a will let us
focus on higher level things that core history storage for the next

Some reasons to use 2a rather than 1.9 or 1.9-rich-root.

For starters, in 2a we've dropped all support for non-rich-root flavour,
letting us finally put behind us a very early design defect in bzr.
Using the 1.9 format would require emacs to migrate past that change -
but its better and more efficient to start with a rich root format
(which 2a is).

Secondly, much of the performance improvements are obtained by using a
better representation for the hash that contains versioned file data in
a revision; 1.9 is _massively_ slower than 2a, and doing the conversion
is inordinately time consuming (even though its simple to do, its
hampered by having poor source format performance).

2a is approximately 1/3 the size on disk for most projects, which also
adds to performance, by requiring less data transfer, less cache
footprint etc. 

And 2a is supported. Its not going to get changed at this point [barring
unforeseen critical issues], and we expect to keep it as-is for the
entire bzr 2.x cycle.

I heartily endorse the use of 2a because it will be better, faster, use
less memory and provide a nicer environment. 

>Who are you talking to?  The list traffic tells a very different
>story.  Mark Shuttleworth (Da Beeg Boos) wrote "Thou Shalt Have But
>One Bazaar 2 Format", and got *lots* of pushback.  AFAIK he gave up;
>the Bazaar devs surely act like that pronunciamento is "inoperative".

Mark has asked us to fix the current UI headaches that have emergerd -
some of which we painted ourselves into. One aspect of that is getting
rid of the huge list of formats, and another is not creating that 
problem again. I don't think Mark has given up his desire for us to fix
this. What he has done is acknowledged is that fixing the UI requires
some care and thought - and that with the dev team focusing on the
plumbing changes to really put performance issues to be the cycles
simply are not there to address the UI before 2.0, *or* we delay 2.0
(which primarily consists of changing the default output format to 2a
and thus giving everyone a radically better experience). I don't think
anyone is keen about delaying 2.0 once we've got all the things in
place for changing the default. That list is pretty short now - one
or two server tweaks, some remaining performance work mainly targeted
at the migration process and finally some assistance for people
migrating.

> Robert Collins is *not* on board on "2a", specifically, looms are not
> included in 2a, which means at least two variants of 2a will be out
> there in the wild for a while.  (Silver lining: looms are usable as a
> local improvement, you don't actually need them on the server for many
> common workflows.) 

Looms have never hooked into the history storage layer (which is what 2a
is all about). Fixing the root cause for the UI friction I refer to above
will likely bring in sufficient core capabilities that loom can stop being
a separate branch type - this has been touched on a few times on the list
I think. If not, I'm fairly sure its in the devnotes branch.

> Aaron Bentley is *not* on board on "2a",
> specifically nested trees have not landed, but they will.  (Emacs
> doesn't care, I think, but many projects want nested trees very badly.
> It will land, not before 2.0, I suspect, but not too long thereafter.
> This will mean a server upgrade, the formats are mutually incompatible
> currently AIUI.)  Four variants.  And counting---don't kid yourself,
> there will be more.

Nested trees are unlikely to hit 2a, they need to be ironed out, transient
code needs to be fully migrated, performance will need to be assessed and
so on. I'm keen to see them completed because they are an important feature,
but its not done till its done.

In short - there is 2a, looms for people that want to use them sit on top of
2a today, and nested trees are not finished yet.

We do need to find a way to gradually deploy such things; all VCS systems do
this, at different rates and in different ways. I'm sure bzr doesn't have the
tradeoffs right yet :(. 

> IMO (and that of the Python devs, see PEPs 374 and 385), Stefan is
> right.  You need to settle on a version that's either Available Now or
> Coming Soon To A GNU/Linux Distro Near You.  People who want to bleed
>can always do that (to) themselves. :-)

I heartily agree - pick a single format and release of bzr, get using
it. That format should be 2a.

-Rob 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Bzr switch

[Daniel Clemente]

El dt, jun 30 2009 a les 04:05, Robert Collins va escriure:
> Secondly, much of the performance improvements are obtained by using a
> better representation for the hash that contains versioned file data in
> a revision; 1.9 is _massively_ slower than 2a, and doing the conversion
> is inordinately time consuming (even though its simple to do, its
> hampered by having poor source format performance).
>
> 2a is approximately 1/3 the size on disk for most projects, which also
> adds to performance, by requiring less data transfer, less cache
> footprint etc. 
>
  This post has benchmarks and gives very good results for 2a, even for OpenOffice.org (260K revisions).
http://bazaarvcs.wordpress.com/2009/06/30/scability-benchmarking/

  Quote from there:

,----
| In a nutshell, 2a with the latest bzr.dev is looking pretty good:
|     * status, diff and commit typically take a second or two
|     * recent history commands take 0.2 seconds
|     * full history log takes around a minute
`----