Locks on the Bzr repository

Locks on the Bzr repository

[Stefan Monnier]

A couple times now, I've had the following kind of message when trying
to commit:

Unable to obtain lock sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/.bzr/branch/lock    
held by jasonr@gnu.org on host wanchan [process #23356]
locked 28 minutes, 51 seconds ago

The first time I thought it was a stale lock so I broke it, but then it
came right back, so I suspect it might be due to something else.
Could it be that bzr grabs the lock when you do "bzr commit" and keep it
held for as long as you're editing the commit message?

If that's indeed the culprit, I think it's a misfeature of Bzr, but at
least we can workaround it by using "bzr commit -m <...>", which I would
then *strongly* recommend.


        Stefan

Re: Locks on the Bzr repository

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> If that's indeed the culprit, I think it's a misfeature of Bzr, but at
> least we can workaround it by using "bzr commit -m <...>", which I would
> then *strongly* recommend.

Even better: use an unbound branch.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Jan Djärv]

Andreas Schwab skrev 2010-08-19 17.52:
> Stefan Monnier<monnier@iro.umontreal.ca>  writes:
>
>> If that's indeed the culprit, I think it's a misfeature of Bzr, but at
>> least we can workaround it by using "bzr commit -m<...>", which I would
>> then *strongly* recommend.
>
> Even better: use an unbound branch.
>

You have to commit upstream sometime.

	Jan D.

Re: Locks on the Bzr repository

[Andreas Schwab]

Jan Djärv <jan.h.d@swipnet.se> writes:

> You have to commit upstream sometime.

Why?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Jan Djärv]

Andreas Schwab skrev 2010-08-19 18.36:
> Jan Djärv<jan.h.d@swipnet.se>  writes:
>
>> You have to commit upstream sometime.
>
> Why?
>

It is no fun fixing bugs and adding features and the just keep it to yourself.
Push has been discuraged on this list and other places several times.

	Jan D.

Re: Locks on the Bzr repository

[Andreas Schwab]

Jan Djärv <jan.h.d@swipnet.se> writes:

> Push has been discuraged on this list and other places several times.

What???

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Jan Djärv]

Andreas Schwab skrev 2010-08-19 23.49:
> Jan Djärv<jan.h.d@swipnet.se>  writes:
>
>> Push has been discuraged on this list and other places several times.
>
> What???
>

We been through this before, search the archives.  Anyway, pull is not faster 
than commit.

	Jan D.

Re: Locks on the Bzr repository

[Andreas Schwab]

Jan Djärv <jan.h.d@swipnet.se> writes:

> We been through this before, search the archives.  Anyway, pull is not
> faster than commit.

A local commit is faster, and the push can then transfer all pending
commits in a single transaction.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Fri, 20 Aug 2010 09:40:43 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> Jan Djärv <jan.h.d@swipnet.se> writes:
> 
> > We been through this before, search the archives.  Anyway, pull is not
> > faster than commit.
> 
> A local commit is faster, and the push can then transfer all pending
> commits in a single transaction.

That's true, but it doesn't necessarily help too much, because pushing
many pending commits moves much more stuff through the wire, and you
need to wait longer.  IOW, the overhead of a small commit is not much
larger, relatively to the diffs, than the overhead of a large commit,
at least for my connection.

Plus, pushing many unrelated commits has a drawback as not showing in
"bzr log" unless you also use the --include-merges (or -n0) switch,
which makes "bzr log" significantly slower.

Worst of all, no progress is being reported regarding turning on
bzr+ssh on savannah.  It's been 9 months since work on this was
supposed to have been started, enough time to deliver a baby, and I
have no idea when (or if) this will happen.  Could those "in the know"
please tell what is holding this back?  Given the information about
the difficulties, maybe someone, whose connection is not so fast and
whose motivation is therefore high, could help.

Re: Locks on the Bzr repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Plus, pushing many unrelated commits has a drawback as not showing in
> "bzr log" unless you also use the --include-merges (or -n0) switch,
> which makes "bzr log" significantly slower.

This has nothing at all to do with pushing.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: jan.h.d@swipnet.se,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 20 Aug 2010 11:13:58 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Plus, pushing many unrelated commits has a drawback as not showing in
> > "bzr log" unless you also use the --include-merges (or -n0) switch,
> > which makes "bzr log" significantly slower.
> 
> This has nothing at all to do with pushing.

Of course, it does.

Re: Locks on the Bzr repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: jan.h.d@swipnet.se,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Fri, 20 Aug 2010 11:13:58 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Plus, pushing many unrelated commits has a drawback as not showing in
>> > "bzr log" unless you also use the --include-merges (or -n0) switch,
>> > which makes "bzr log" significantly slower.
>> 
>> This has nothing at all to do with pushing.
>
> Of course, it does.

???? In which way?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: jan.h.d@swipnet.se,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 20 Aug 2010 13:35:05 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andreas Schwab <schwab@linux-m68k.org>
> >> Cc: jan.h.d@swipnet.se,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> >> Date: Fri, 20 Aug 2010 11:13:58 +0200
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> > Plus, pushing many unrelated commits has a drawback as not showing in
> >> > "bzr log" unless you also use the --include-merges (or -n0) switch,
> >> > which makes "bzr log" significantly slower.
> >> 
> >> This has nothing at all to do with pushing.
> >
> > Of course, it does.
> 
> ???? In which way?

IIUC, if you resync with the master repo with "bzr merge", your local
commits will appear on a branch when you push.

Re: Locks on the Bzr repository

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> IIUC, if you resync with the master repo with "bzr merge", your local
> commits will appear on a branch when you push.

No, your local commits will appear on the leftmost side of the DAG. But
some commits (precisely those you merged on your local branch) that
previously were on the leftmost side of the DAG are displaced to the
right, which is very bad. See my other post.

Re: Locks on the Bzr repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> IIUC, if you resync with the master repo with "bzr merge", your local
> commits will appear on a branch when you push.

Did I write "bzr merge" anywhere?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: jan.h.d@swipnet.se,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 20 Aug 2010 15:29:40 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > IIUC, if you resync with the master repo with "bzr merge", your local
> > commits will appear on a branch when you push.
> 
> Did I write "bzr merge" anywhere?

No.  But if you don't have that workflow in mind, then we will have no
common language until you describe in more detail the workflow you
have in mind.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Plus, pushing many unrelated commits has a drawback as not showing
 > in "bzr log" unless you also use the --include-merges (or -n0)
 > switch, which makes "bzr log" significantly slower.

No.  Push simply synchronizes the histories, and IIRC to push in bzr
you must be a superset of the remote repo's history.  So what matters
to the bzr log view is how you manage your local repo, not whether you
push or commit via a bound branch.  You may need to rebase if somebody
sneaks in a commit before you start your push, but I would imagine you
normally need less locking and a lot less time if the push succeeds.

It's true that if you only ever use bound branches, all your commits
must end up on the mainline, and that may be the best strategy for
some people.  But other people like to commit more frequently and/or
offline.

Re: Locks on the Bzr repository

[Andreas Schwab]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> But other people like to commit more frequently and/or offline.

That's the whole point of a distriuted VCS anyway.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Andreas Schwab writes:
 > "Stephen J. Turnbull" <stephen@xemacs.org> writes:

 > > But other people like to commit more frequently and/or offline.

 > That's the whole point of a distriuted VCS anyway.

Not quite the whole point, at least compared to CVS.

In any case, that point seems to be totally lost on certain
influential Emacs developers, so it bears repetition.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Andreas Schwab <schwab@linux-m68k.org>,
>     jan.h.d@swipnet.se,
>     monnier@iro.umontreal.ca,
>     emacs-devel@gnu.org
> Date: Fri, 20 Aug 2010 19:15:38 +0900
> 
> Eli Zaretskii writes:
> 
>  > Plus, pushing many unrelated commits has a drawback as not showing
>  > in "bzr log" unless you also use the --include-merges (or -n0)
>  > switch, which makes "bzr log" significantly slower.
> 
> No.  Push simply synchronizes the histories, and IIRC to push in bzr
> you must be a superset of the remote repo's history.  So what
> matters to the bzr log view is how you manage your local repo

If I ever only "bzr merge" from the remote repository, won't my pushes
to it cause my local commits appear on a branch, and the point of push
appear as a merge to mainline?  If so, that's what I meant.

> You may need to rebase if somebody sneaks in a commit before you
> start your push

With the rate of commits to the Emacs repository, the chances that
someone "sneaks in a commit before you start your push" are 100%.  We
have committers in all time zones, so there's no time that you are
safe.

> but I would imagine you normally need less locking and a lot less
> time if the push succeeds.

This needs testing and measuring; I won't believe it until I actually
see it.  The time to push is still governed by network traffic, which
takes most of the time, about 90%, of a commit as well.

And even if it is true that this is faster, what you describe hints on
a much more complex workflow that needs a much more diligent user who
knows much more about bzr than most contributors.  Any blunder there,
and you need to work much harder and at least partially by hand to
restore the situation where your branch is "a superset of the remote
repo's history".  So this workflow still has disadvantages, even if it
faster (which I very much doubt).

> It's true that if you only ever use bound branches, all your commits
> must end up on the mainline, and that may be the best strategy for
> some people.  But other people like to commit more frequently and/or
> offline.

Me too, but having a separate local branch and merging locally with a
bound branch has all the advantages of the frequent fast commits, and
only one disadvantage -- the commits appear on a branch (unless I
rebase).  The workflow is much simpler, though.  That's why this is
what I do when I work on something that is not a 5-min job.

Re: Locks on the Bzr repository

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

> Me too, but having a separate local branch and merging locally with a
> bound branch has all the advantages of the frequent fast commits, and
> only one disadvantage -- the commits appear on a branch (unless I
> rebase).  The workflow is much simpler, though.  That's why this is
> what I do when I work on something that is not a 5-min job.

Just to clarify:

merge+push has the side effect of twisting the DAG. This means that the
left arm becomes the right, and the right the left. As Eli says, you'll
need to use `bzr log -n0' for seeing the commits that previously were
visible with `bzr log' (sans the -n0) In general, the DAG becomes a
mess. But, worse of all, revision numbers change, because revision
numbering depends on how far from the left is the branch of the DAG that
contains the commit. What now is revision 1000 after a merge+push can be
revision 995.3.2, and some time later 998.5.3.2. This would force us to
use the awkward bzr revision ids for referring revisions.

The Right Thing if you want to accumulate several unrelated local
changes and send them upstream on one batch is to rebase+commit, which
appends your commits to the tip of upstream's branch.

This is not very different from Git. On Git it is not so bad to
merge+push for incoporating your feature branch into mainline (it twists
the DAG sides too, but Git does not make big differences among them.)
However, for sending a series of unrelated commits, the Right Thing on
Git is to rebase+push, which is mostly equivalent to bzr's
rebase+commit.

All this was previously discussed on the list. And we had already some
instances of revision renumbering, so the configuration of upstream repo
was changed for rejecting the operations that would twist the DAG.

Re: Locks on the Bzr repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> With the rate of commits to the Emacs repository, the chances that
> someone "sneaks in a commit before you start your push" are 100%.

Is it?  I almost never have that problem.  Moreover, this has nothing to
do with bound vs. unbound, since you are racing with other commiters in
any case.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>,  jan.h.d@swipnet.se,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 20 Aug 2010 15:33:31 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > With the rate of commits to the Emacs repository, the chances that
> > someone "sneaks in a commit before you start your push" are 100%.
> 
> Is it?  I almost never have that problem.

Maybe you are a faster programmer than I am.  After "bzr up" that
brings new code, I need to build Emacs and see that nothing breaks due
to my changes.  Sometimes I need to build on more than one platform.
It takes time, at least here.

> Moreover, this has nothing to do with bound vs. unbound

Did I say it had something to do with this?

Re: Locks on the Bzr repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Did I say it had something to do with this?

Yes.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > If I ever only "bzr merge" from the remote repository, won't my
 > pushes to it cause my local commits appear on a branch, and the
 > point of push appear as a merge to mainline?  If so, that's what I
 > meant.

Sure.  So don't do that.  One (probably for values of "one" != "eliz")
can pull instead, and rebase when necessary to get that to succeed.
Or one can use a separate branch for each task.  There are lots of
strategies; you don't have to use them if you don't want to.  But some
people would rather spend 5 minutes fiddling with a rebase once a week
(which can be done at your convenience) or so than ever spend 50
minutes waiting for a commit to finish (which blocks you from doing
any other commit, if you depend on lightweight checkouts from
upstream).

 > With the rate of commits to the Emacs repository, the chances that
 > someone "sneaks in a commit before you start your push" are 100%.

That's an exaggeration.  It might be quite high, but if it's really
that high that pushes never succeed, any integration where developers
decided asynchronously when to commit to the public repo (including
both push and commits in bound branches) is going to run into problems.

 > We have committers in all time zones, so there's no time that you
 > are safe.

That depends on one's definition of safe.  My definition of safe is
"my commit is safely in a repository with a known relationship to the
upstream", I can commit any time the fit takes me, and I can push when
I feel like dealing with the costs of cooperation like merge
conflicts.

 > > but I would imagine you normally need less locking and a lot less
 > > time if the push succeeds.

 > This needs testing and measuring; I won't believe it until I
 > actually see it.  The time to push is still governed by network
 > traffic, which takes most of the time, about 90%, of a commit as
 > well.

Not if you count *time developer is blocked* instead of *time from
start of operation to end of operation*.  Push is asynchronous; with a
DVCS you don't even need to care if it succeeds, you just work on
another branch.  If it succeeded, you're done.  If not, you make
adjustments and try again later, at your convenience.  (What those
adjustments are depends on the exact workflow.)

 > And even if it is true that this is faster, what you describe hints
 > on a much more complex workflow that needs a much more diligent
 > user who knows much more about bzr than most contributors.

Sure.  So what?  I don't need to defend a deprecation of "checkout
--lightweight".  I don't like it, but I haven't ever suggested that it
be forbidden.  *You* need to defend your deprecation of "push."

 > Any blunder there, and you need to work much harder and at least
 > partially by hand to restore the situation where your branch is "a
 > superset of the remote repo's history".  So this workflow still has
 > disadvantages,

All workflows have disadvantages.  Big deal.  That's not a reason for
a general deprecation of "bzr push".

 > > It's true that if you only ever use bound branches, all your
 > > commits must end up on the mainline, and that may be the best
 > > strategy for some people.  But other people like to commit more
 > > frequently and/or offline.

 > Me too, but having a separate local branch and merging locally with
 > a bound branch has all the advantages of the frequent fast commits,
 > and only one disadvantage -- the commits appear on a branch (unless
 > I rebase).

No, it has another disadvantage, which is that you can hang up on
concurrent commits by other developers, just as with the push
strategy.  Perhaps it's a little less apparent work because you're
only blocked on merges with conflicts, but once again you're blocked
until your commit succeeds.

It's also somewhat dangerous to just merge everything; a diligent
developer should treat a merge in such case as "my commit needs review
in the new context before committing", really, and what you're doing
is short-circuiting that process.  (Linus famously flamed Dave Miller
for this.)  Any merge (which includes rebasing, which is what davem
was doing) implies your code is now untested until you rerun the
tests.  So if you merge into a bound branch, you're automating commits
of untested code to the public repo.  That may be OK with you (after
all, the test results are only a little bit stale, they're probably
not growing mold and mushrooms yet), but that is what you're doing.

I don't expect this to change your mind about *your* workflows.  But
there are good arguments for other workflows, there are definite
weaknesses in the ones you like as well, and I don't think it's a good
idea to deprecate push-based workflows for those who want to use them.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: jan.h.d@swipnet.se,
>     schwab@linux-m68k.org,
>     monnier@iro.umontreal.ca,
>     emacs-devel@gnu.org
> Date: Fri, 20 Aug 2010 22:44:28 +0900
> 
> Eli Zaretskii writes:
> 
>  > If I ever only "bzr merge" from the remote repository, won't my
>  > pushes to it cause my local commits appear on a branch, and the
>  > point of push appear as a merge to mainline?  If so, that's what I
>  > meant.
> 
> Sure.  So don't do that.  One (probably for values of "one" != "eliz")
> can pull instead, and rebase when necessary to get that to succeed.

Please show this workflow in more detail.  Without the details, it's
hard to tell what you suggest.  You could be thinking about things
possible with git, but not with bzr; it happened in the past.

And please drop the ad hominem, you are (again) incorrectly assuming
that I don't use distributed workflows.  You can see that this is not
so if you look at my commits in "bzr log" .

> Or one can use a separate branch for each task.  There are lots of
> strategies; you don't have to use them if you don't want to.

I do want and do use some of them.  Please don't mistake a critical
opinion for rejection.  Instead, back up your opinions by valid
arguments, and maybe I will be able to actually learn something from
you.

> But some
> people would rather spend 5 minutes fiddling with a rebase once a week
> (which can be done at your convenience) or so than ever spend 50
> minutes waiting for a commit to finish (which blocks you from doing
> any other commit, if you depend on lightweight checkouts from
> upstream).

I don't use lightweight checkouts.  I use one bound branch and 2 or 3
local branches.  I have no problems with local commits, which I use a
lot.  The problem happens when I need to commit to the master
repository.  I need to do that from time to time, because my local
commits are not available to others, so if I fix bugs, my fixes will
not be good to anyone until I commit to upstream.  And doing so is
painfully slow, and there seems to be no way around it.  On top of
that, sometimes a commit fails, because someone holds a lock or
because some other commit was "sneaked" in in the meantime.  So I need
to watch the commit and make sure it succeeds, or start another one if
it doesn't.  So please don't tell me this is asynchronous, because it
isn't, at least not entirely.

>  > With the rate of commits to the Emacs repository, the chances that
>  > someone "sneaks in a commit before you start your push" are 100%.
> 
> That's an exaggeration.

How do you know?  How frequently do you commit to the Emacs repo?

>  > > but I would imagine you normally need less locking and a lot less
>  > > time if the push succeeds.
> 
>  > This needs testing and measuring; I won't believe it until I
>  > actually see it.  The time to push is still governed by network
>  > traffic, which takes most of the time, about 90%, of a commit as
>  > well.
> 
> Not if you count *time developer is blocked* instead of *time from
> start of operation to end of operation*.  Push is asynchronous; with a
> DVCS you don't even need to care if it succeeds, you just work on
> another branch.  If it succeeded, you're done.  If not, you make
> adjustments and try again later, at your convenience.

You are talking from the POV of someone who has a lot of time on his
hands, or has a fast enough connection to make the delay
insignificant.  By contrast, I have maybe a few hours during a weekend
to work on Emacs, with maybe half an hour on weekdays.  Every delay
counts, and there's no "at my convenience".

> *You* need to defend your deprecation of "push."

I don't deprecate "push", so I have nothing to defend.

> All workflows have disadvantages.  Big deal.

No big deal.  But the disadvantages of each workflow need to be well
understood before making a decision which one to use.  And the
considerations might well be different given one's personal
circumstances and connectivity.  Careful examination of the advantages
and disadvantages does not constitute deprecation, and should not be
treated with derogation.

>  That's not a reason for a general deprecation of "bzr push".

I don't deprecate "push".

>  > > It's true that if you only ever use bound branches, all your
>  > > commits must end up on the mainline, and that may be the best
>  > > strategy for some people.  But other people like to commit more
>  > > frequently and/or offline.
> 
>  > Me too, but having a separate local branch and merging locally with
>  > a bound branch has all the advantages of the frequent fast commits,
>  > and only one disadvantage -- the commits appear on a branch (unless
>  > I rebase).
> 
> No, it has another disadvantage, which is that you can hang up on
> concurrent commits by other developers, just as with the push
> strategy.  Perhaps it's a little less apparent work because you're
> only blocked on merges with conflicts, but once again you're blocked
> until your commit succeeds.

As long as you commit to the master repo, you will become blocked
sometimes.  And if you think that my development is blocked while the
commit grinds away, then you are simply wrong, because I have
additional branches where I can and do develop in the meantime.  But
the slow commits that sometimes fail after many seconds if not minutes
are an annoyance anyhow, and no amount of preaching the "push" ways
can do anything about it.

> It's also somewhat dangerous to just merge everything; a diligent
> developer should treat a merge in such case as "my commit needs review
> in the new context before committing", really

Exactly.  And as I explained in another mail in this thread, this
calls for testing after I merge, which is one factor why the chances
are high to have some other commit sneaked in before I commit.

> and what you're doing is short-circuiting that process.

What am I doing that short-circuits this process?

> (Linus famously flamed Dave Miller
> for this.)  Any merge (which includes rebasing, which is what davem
> was doing) implies your code is now untested until you rerun the
> tests.  So if you merge into a bound branch, you're automating commits
> of untested code to the public repo.  That may be OK with you (after
> all, the test results are only a little bit stale, they're probably
> not growing mold and mushrooms yet), but that is what you're doing.

Don't ass-u-me that I'm committing without testing after the merge.

> I don't expect this to change your mind about *your* workflows.

Looks like you have no idea at all what _is_ my mindset regarding the
possible workflows.  Perhaps you think you are talking to someone
else.

> But there are good arguments for other workflows, there are definite
> weaknesses in the ones you like as well, and I don't think it's a
> good idea to deprecate push-based workflows for those who want to
> use them.

I don't deprecate "push.

Re: Locks on the Bzr repository

[Uday S Reddy]

Perhaps I can sneak in a few comments while Stephen is offline.  I have found 
this debate quite enlightening and it helped me understand the limits of 
"distributed" VCS.  (I use unbound local branches in working on VM and always 
wondered why you guys use bound branches.  I am beginning to understand a 
little bit now.)

On 8/20/2010 4:19 PM, Eli Zaretskii wrote:

>>
>> Sure.  So don't do that.  One (probably for values of "one" != "eliz")
>> can pull instead, and rebase when necessary to get that to succeed.
>
> Please show this workflow in more detail.  Without the details, it's
> hard to tell what you suggest.  You could be thinking about things
> possible with git, but not with bzr; it happened in the past.

I think one needs to do a pull before each commit, to be safe.  (Seems to be 
the same as update-commit cycle for bound branches.)

If you have already done a series of commits and lost the opportunity to pull, 
then you have to rebase.

If even that seems too far out, then you have to make your changes into a 
separate branch and merge it in later.

> I don't use lightweight checkouts.  I use one bound branch and 2 or 3
> local branches.  I have no problems with local commits, which I use a
> lot.  The problem happens when I need to commit to the master
> repository.  I need to do that from time to time, because my local
> commits are not available to others, so if I fix bugs, my fixes will
> not be good to anyone until I commit to upstream.

It seems to me that committing to the bound branch is essentially like push. 
(Somebody said you were "discouraging" push.  I don't see it.  Your every 
commit is a push.)

However, we get to use our local mirror of the trunk as if it were a local 
branch.  We can commit to it without affecting anybody else and do a public 
commit only in the end.  We can commit to it as often as we please, keep a 
careful log, do enough testing etc. And, this local branch can magically turn 
back into a proper mirror as soon as we do a rebase-push.  You can't get this 
with bound branches.  Your local branches have to be necessarily separate 
branches, whose histories will appear as merges in the mainline.

We use separate local branches as well, when they are appropriate.  But we are 
not forced to use them all the time.  (Generally, I am happy to use the main 
branch for a few days worth of work where I can see the end clearly.  I use 
separate branches for things that might take a few weeks, or things of a more 
open-ended nature.)

   And doing so is
> painfully slow, and there seems to be no way around it.  On top of
> that, sometimes a commit fails, because someone holds a lock or
> because some other commit was "sneaked" in in the meantime.  So I need
> to watch the commit and make sure it succeeds, or start another one if
> it doesn't.  So please don't tell me this is asynchronous, because it
> isn't, at least not entirely.

Seems to me that you are reinforcing Stephen's point.  With bound branches, 
your branch is locked up until the commit goes through.  You can't do anything 
while you have uncommitted changes in your source.  With unbound branches, we 
can continue working on the source even when push is running in the background, 
because the source tree doesn't have any uncommitted changes.  We can also give 
up on the push if necessary and continue committing to the branch.  The 
advantage seems quite clear to me.

> No big deal.  But the disadvantages of each workflow need to be well
> understood before making a decision which one to use.

I can't believe that you guys really understood the disadvantages of your 
current workflow before settling on it.

The most illuminating revelation I found in Stephen's post is that, with bound 
branches, every commit is a merge.  Most of the time, it is untested and 
unsafe.  This is the biggest disadvantage of all, if you ask me.

It also occurs to me that, by asking everybody to use bound branches, you have 
massively increased the contention on your public repo and the server.  That is 
making your problems worse.

Cheers,
Uday

Re: Locks on the Bzr repository

[Jan Djärv]

Uday S Reddy skrev 2010-08-21 00.41:

> Seems to me that you are reinforcing Stephen's point. With bound branches,
> your branch is locked up until the commit goes through. You can't do anything
> while you have uncommitted changes in your source. With unbound branches, we
> can continue working on the source even when push is running in the
> background, because the source tree doesn't have any uncommitted changes. We
> can also give up on the push if necessary and continue committing to the
> branch. The advantage seems quite clear to me.

You are ignoring the fact that work usually doesn't happen in the bound 
branch, but in a separate task branch.  We can continue to work there while 
the bound branch commits.  I don't see much difference.

	Jan D.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sat, 21 Aug 2010 10:36:25 +0200
> From: Jan Djärv <jan.h.d@swipnet.se>
> Cc: emacs-devel@gnu.org
> 
> Uday S Reddy skrev 2010-08-21 00.41:
> 
> > Seems to me that you are reinforcing Stephen's point. With bound branches,
> > your branch is locked up until the commit goes through. You can't do anything
> > while you have uncommitted changes in your source. With unbound branches, we
> > can continue working on the source even when push is running in the
> > background, because the source tree doesn't have any uncommitted changes. We
> > can also give up on the push if necessary and continue committing to the
> > branch. The advantage seems quite clear to me.
> 
> You are ignoring the fact that work usually doesn't happen in the bound 
> branch, but in a separate task branch.  We can continue to work there while 
> the bound branch commits.

Actually, I see no reason not to continue working even in the bound
branch that is being committed.  There's nothing at all to prevent
that, since Bazaar takes note of the files it commits and their
contents _before_ it sends changes upstream.  That is why you cannot
make changes after launching "bzr ci" and hope for them to be included
in the changeset.  (This is unlike CVS, where you could make changes
as long as the particular file wasn't sent upstream by "cvs ci".)

Re: Locks on the Bzr repository

[Jan Djärv]

Eli Zaretskii skrev 2010-08-21 11.08:
>> Date: Sat, 21 Aug 2010 10:36:25 +0200
>> From: Jan Djärv<jan.h.d@swipnet.se>
>> Cc: emacs-devel@gnu.org
>>
>> Uday S Reddy skrev 2010-08-21 00.41:
>>
>>> Seems to me that you are reinforcing Stephen's point. With bound branches,
>>> your branch is locked up until the commit goes through. You can't do anything
>>> while you have uncommitted changes in your source. With unbound branches, we
>>> can continue working on the source even when push is running in the
>>> background, because the source tree doesn't have any uncommitted changes. We
>>> can also give up on the push if necessary and continue committing to the
>>> branch. The advantage seems quite clear to me.
>>
>> You are ignoring the fact that work usually doesn't happen in the bound
>> branch, but in a separate task branch.  We can continue to work there while
>> the bound branch commits.
>
> Actually, I see no reason not to continue working even in the bound
> branch that is being committed.  There's nothing at all to prevent
> that, since Bazaar takes note of the files it commits and their
> contents _before_ it sends changes upstream.  That is why you cannot
> make changes after launching "bzr ci" and hope for them to be included
> in the changeset.  (This is unlike CVS, where you could make changes
> as long as the particular file wasn't sent upstream by "cvs ci".)

This is correct.  However any bzr operation like a simple C-x v = fails 
because bzr is locked by the commit (it doesn't seem to do read-only locks). 
That is the main disadvantage that makes me go back to the task branch.

	Jan D.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sat, 21 Aug 2010 12:30:44 +0200
> From: Jan Djärv <jan.h.d@swipnet.se>
> CC: u.s.reddy@cs.bham.ac.uk, emacs-devel@gnu.org
> 
> > Actually, I see no reason not to continue working even in the bound
> > branch that is being committed.  There's nothing at all to prevent
> > that, since Bazaar takes note of the files it commits and their
> > contents _before_ it sends changes upstream.  That is why you cannot
> > make changes after launching "bzr ci" and hope for them to be included
> > in the changeset.  (This is unlike CVS, where you could make changes
> > as long as the particular file wasn't sent upstream by "cvs ci".)
> 
> This is correct.  However any bzr operation like a simple C-x v = fails 
> because bzr is locked by the commit (it doesn't seem to do read-only locks). 
> That is the main disadvantage that makes me go back to the task branch.

I agree that this is a disadvantage, but is it really that
significant?  How much do you use bzr operations while developing?
Most of development is not about bzr, it's about editing, compiling,
and testing.  I only need bzr operations when I'm almost done with
development, like to see what files are changed or compare them with
some other version.  No matter how slow is "bzr ci" upstream, the
development cycle is much slower, at least for me.  It can only be
faster if you fix very simple bugs, like typos.

YMMV, but this disadvantage doesn't sound a reason good enough to make
a cardinal decision to do all the work from a task branch.

Re: Locks on the Bzr repository

[Jan Djärv]

Eli Zaretskii skrev 2010-08-21 12.41:
>> Date: Sat, 21 Aug 2010 12:30:44 +0200
>> From: Jan Djärv<jan.h.d@swipnet.se>

>> This is correct.  However any bzr operation like a simple C-x v = fails
>> because bzr is locked by the commit (it doesn't seem to do read-only locks).
>> That is the main disadvantage that makes me go back to the task branch.
>
> I agree that this is a disadvantage, but is it really that
> significant?  How much do you use bzr operations while developing?
> Most of development is not about bzr, it's about editing, compiling,
> and testing.  I only need bzr operations when I'm almost done with
> development, like to see what files are changed or compare them with
> some other version.  No matter how slow is "bzr ci" upstream, the
> development cycle is much slower, at least for me.  It can only be
> faster if you fix very simple bugs, like typos.
>
> YMMV, but this disadvantage doesn't sound a reason good enough to make
> a cardinal decision to do all the work from a task branch.

It depends on your mode of programming I think.  Most of the time it doesn't 
matter.  But sometimes while searching for a bug, I put debug messages in 
several files (more often so when I am in an area I have little knowledge of). 
  Or I make what I think are the correct changes, but then it turns out I was 
mistaken.  So I want to remove some of the changes.  C-x v = or C-x v d helps 
me find them.  For new stuff I don't use bzr that much, if at all.

	Jan D.

Re: Locks on the Bzr repository

[Thien-Thi Nguyen]

() Eli Zaretskii <eliz@gnu.org>
() Sat, 21 Aug 2010 13:41:02 +0300

   Most of development is not about bzr, it's about editing,
   compiling, and testing.

Depends on the area of development.  When in janitor mode, it seems
to me most development is about about understanding previous development.
(Too bad there is no M-x empathize RET j.r.hacker RET 1995 RET.)

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Thien-Thi Nguyen writes:
 > () Eli Zaretskii <eliz@gnu.org>
 > () Sat, 21 Aug 2010 13:41:02 +0300
 > 
 >    Most of development is not about bzr, it's about editing,
 >    compiling, and testing.
 > Depends on the area of development.  When in janitor mode, it seems
 > to me most development is about about understanding previous
 > development.  (Too bad there is no M-x empathize RET j.r.hacker RET
 > 1995 RET.)

bzr alias empathize annotate ?

Re: Locks on the Bzr repository

[Thien-Thi Nguyen]

() "Stephen J. Turnbull" <stephen@xemacs.org>
() Sun, 22 Aug 2010 16:39:46 +0900

   bzr alias empathize annotate ?

I suppose incomplete information is better than none.

Re: Locks on the Bzr repository

[Uday S Reddy]

Jan Djärv writes:

> You are ignoring the fact that work usually doesn't happen in the bound 
> branch, but in a separate task branch.  We can continue to work there while 
> the bound branch commits.  I don't see much difference.

If there isn't "much difference" then why is push being discouraged?
You are arguing both ways, it seems to me.

If you use separate task branches then, yes, it doesn't make any
difference whether you use a bound branch or unbound.  So, there is no
reason to prefer one over the other.

If you want to work on the main branch rather than task branches, then
it does make an important difference, as Stephen and I have pointed
out.  Here, an unbound main branch is a lot more flexible and it will
reduce the contention on the central repo.

But this part of the thread is in reponse to Eli's post on the 20th,
especially his comment:

Eli Zaretskii writes:

> Plus, pushing many unrelated commits has a drawback as not showing in
> "bzr log" unless you also use the --include-merges (or -n0) switch,
> which makes "bzr log" significantly slower.

If Eli is trying to avoid levels in histories, then he *must be*
working on the main branch.  And, he is stung by the bound main
branch.  Using an unbound main branch will immediately improve
things.

Note that he also raised a logical problem.  If you open a separate
branch for your work, then you don't want to put many unrelated
commits in it.  That will make it appear as if they were related
commits at the top level.  Using bound branches will encourage people
to put unrelated commits in task branches.

Whichever way I look at it, I don't see any upside to using bound
branches, but plenty of downside.  So, I am surprised that emacs-devs
choose this mode of operation.

Cheers,
Uday

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sat, 21 Aug 2010 10:38:46 +0100
> From: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>
> Cc: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>, emacs-devel@gnu.org
> 
> Jan Djärv writes:
> 
> > You are ignoring the fact that work usually doesn't happen in the bound 
> > branch, but in a separate task branch.  We can continue to work there while 
> > the bound branch commits.  I don't see much difference.
> 
> If there isn't "much difference" then why is push being discouraged?

The only place on the wiki which could be considered as "discouraging"
push is this:

  It might occur to you to save some effort by just doing bzr push
  directly to the upstream master from inside the TASKNAME branch:

	cd $DEVHOME/emacs/TASKNAME
	bzr push sftp://USERNAME@bzr.savannah.gnu.org/srv/bzr/emacs/trunk/

  *Do not do this* -- it can cause history to be displayed in a strange
  way in the upstream master, any mirrors or branches of it, and your own
  branch later. Search for the word “hidden” in
     http://lists.gnu.org/archive/html/emacs-devel/2009-11/msg01021.html
  for more details.

IIRC, this was written by Stephen and Karl, and I don't see anything
wrong with this warning.  Do you?

> If you want to work on the main branch rather than task branches, then
> it does make an important difference, as Stephen and I have pointed
> out.  Here, an unbound main branch is a lot more flexible and it will
> reduce the contention on the central repo.

As I wrote in an earlier post, I don't see the extra flexibility.  I'm
awaiting your response to that.

> > Plus, pushing many unrelated commits has a drawback as not showing in
> > "bzr log" unless you also use the --include-merges (or -n0) switch,
> > which makes "bzr log" significantly slower.
> 
> If Eli is trying to avoid levels in histories, then he *must be*
> working on the main branch.

I'm trying to keep the history linear whenever possible, yes.  If you
think this is not a good idea on its own right, please tell why.

> And, he is stung by the bound main branch.  Using an unbound main
> branch will immediately improve things.

Again, I don't see how am I "stung" and how an unbound branch will
improve things.

> Note that he also raised a logical problem.  If you open a separate
> branch for your work, then you don't want to put many unrelated
> commits in it.  That will make it appear as if they were related
> commits at the top level.  Using bound branches will encourage people
> to put unrelated commits in task branches.

If you do ALL of the work in a task branch, yes.  But I don't think
that's the right workflow, and that's not what I use.  I use task
branches for 2 kinds of jobs:

  . prolonged development of features

  . forensics: debugging some issue that needs switching to earlier
    versions, e.g. with "bzr bisect" or "bzr revert -rNNN" (later I
    get rid of all the traces of these revert commands by pulling from
    the main branch with "bzr pull --overwrite")

> Whichever way I look at it, I don't see any upside to using bound
> branches, but plenty of downside.

Well, would you please list those downsides together?  They must have
been lost in the argument, because I don't see them.  At most I've
seen a single downside you cited -- that further development is
somehow stuck until the commit from the bound branch finishes.  As I
wrote, I think this claim is simply false.

Re: Locks on the Bzr repository

[Jan Djärv]

Uday S Reddy skrev 2010-08-21 11.38:
> Jan Djärv writes:
>
>> You are ignoring the fact that work usually doesn't happen in the bound
>> branch, but in a separate task branch.  We can continue to work there while
>> the bound branch commits.  I don't see much difference.
>
> If there isn't "much difference" then why is push being discouraged?

Something about history I think.

> You are arguing both ways, it seems to me.

I'm arguing that push doesn't buy you anything w.r.t. to speed when sending 
changes to Savannah.  If by both ways you mean "push is slow" and "commit is 
slow" you are correct.  We can just say "bzr is slow" and be done with it.


>
> Whichever way I look at it, I don't see any upside to using bound
> branches, but plenty of downside.  So, I am surprised that emacs-devs
> choose this mode of operation.
>

It was recommended here: http://www.emacswiki.org/emacs/BzrForEmacsDevs. 
Nobody has stepped up and explained in this detail a better way.

	Jan D.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sat, 21 Aug 2010 12:37:45 +0200
> From: Jan Djärv <jan.h.d@swipnet.se>
> Cc: emacs-devel@gnu.org
> 
> I'm arguing that push doesn't buy you anything w.r.t. to speed when sending 
> changes to Savannah.  If by both ways you mean "push is slow" and "commit is 
> slow" you are correct.  We can just say "bzr is slow" and be done with it.

100% agreement.

Re: Locks on the Bzr repository

[Uday S Reddy]

Jan Djärv writes:

> > You are arguing both ways, it seems to me.
> 
> I'm arguing that push doesn't buy you anything w.r.t. to speed when sending 
> changes to Savannah.  If by both ways you mean "push is slow" and "commit is 
> slow" you are correct.  We can just say "bzr is slow" and be done with it.

If you use unbound branches, push is slow but commit is not slow.
Separating push from commit helps you in that, you can do push less
often.

With bound branches, push = commit.  So, you are stuck with the
slowness.

By arguing "both ways", I mean that you want to argue that bound
branches are better.  But, when pushed, the best you can say is that
they are no worse.  Nobody has pointed out anything that makes them
better.

And the "no worse" argument only holds if you exclusively work on
separate task branches.  If you want to work on the main branch and
commit it frequently, like Eli wants to do for bug fixes, then bound
branches are worse because they force you to synchronize for each
commit. 

Cheers,
Uday

Re: Locks on the Bzr repository

[Jan Djärv]

Uday S Reddy skrev 2010-08-21 14.51:
> Jan Djärv writes:
>
>>> You are arguing both ways, it seems to me.
>>
>> I'm arguing that push doesn't buy you anything w.r.t. to speed when sending
>> changes to Savannah.  If by both ways you mean "push is slow" and "commit is
>> slow" you are correct.  We can just say "bzr is slow" and be done with it.
>
> If you use unbound branches, push is slow but commit is not slow.

Duh.  I'm only talking about when stuff goes to Savannah.  I know committing 
locally is much faster, I do it all the time.

> Separating push from commit helps you in that, you can do push less
> often.

No it don't.  When a bug fix is done, or a feature is done, I want it to go to 
Savannah.  Push doesn't help here, when it is time, it is time.

> By arguing "both ways", I mean that you want to argue that bound
> branches are better.

You are putting words in my mouth.  I only said for the same amount of data to 
be sent upstream, push and commit is equally slow.

	Jan D.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sat, 21 Aug 2010 13:51:38 +0100
> From: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>
> Cc: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>, emacs-devel@gnu.org
> 
> By arguing "both ways", I mean that you want to argue that bound
> branches are better.  But, when pushed, the best you can say is that
> they are no worse.  Nobody has pointed out anything that makes them
> better.

The only advantage of a bound branch is that the workflow is simpler.

> If you want to work on the main branch and commit it frequently,
> like Eli wants to do for bug fixes, then bound branches are worse
> because they force you to synchronize for each commit.

You cannot push without synchronizing anyway, or else you will mess up
the upstream history.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > If you want to work on the main branch and commit it frequently,
 > > like Eli wants to do for bug fixes, then bound branches are worse
 > > because they force you to synchronize for each commit.

 > You cannot push without synchronizing anyway, or else you will mess
 > up the upstream history.

Sure, but as long as you have arranged your local branch so that its
mainline commits all belong on the upstream mainline, you synchronize
once *per push* rather than once *per commit*.  Synchronization (done
correctly) is expensive because it involves a build-and-test cycle.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Jan Djärv writes:

 > > Whichever way I look at it, I don't see any upside to using bound
 > > branches, but plenty of downside.  So, I am surprised that
 > > emacs-devs choose this mode of operation.
 > 
 > It was recommended here:
 > http://www.emacswiki.org/emacs/BzrForEmacsDevs. Nobody has stepped
 > up and explained in this detail a better way.

I'm beginning to remember what the discussion was.  (Stefan's right,
life would be a lot easier if Emacs just switches to git.)

The basic problem of distributed development is that there is no
physical mainline (this is *why* conflicts can occur; even with a
centralized VCS, for this purpose we can consider multiple workspaces
as virtual -- unrecorded -- branches).  However, most projects are
organized around a mainline (or a small number of mainlines), and of
course for products with a warrantee, the supported versions
implicitly define a mainline in practice.  Darcs completely ignores
the concept of mainline, by viewing a version as a set (not a sequence
or even DAG) of abstract patches, and adding only minimal
dependencies.  git, hg, and bzr view versions as nodes in a graph, the
history DAG.  Traversing the DAG (for example, to produce a log)
requires choices in the presence of branching and merging (if there
are no merges, AFAIK all DVCSes ignore off-current-branch commits in
log and other branch-tracing operations like annotate).

git and hg of course must choose a traversal algorithm (or more
abstractly a way of converting the DAG to a total order), but this is
considered an arbitrary choice (more or less).  Aside from arbitrary
ordering decisions with respect to parallel branches in the DAG, they
only distinguish between merge nodes (ie, those with multiple parents)
and non-merge nodes.  bzr, OTOH, made a deliberate decision to have a
distinguished mainline, actually, a hierarchy of them, indicated by
the order of parents of each node.

Another important design decision in bzr (similar to hg and darcs) is
that the basic model is workspace == branch == repository.  (git
denied this model from the get-go.  git was always designed to manage
a general DAG and from the very beginning allowed different workspaces
to share an object database.)  The idea is to simplify life for the
user by having one active head per workspace, and the user switches
branches by cd'ing to the workspace with the appropriate version
checked out.  However, having only one active head interacts with the
distinguished mainline in, IMO, an unfortunate way: every new commit
is automatically on the mainline!  That is, the mainline is *workspace
[1]relative*.

Having each workspace contain a mainline means that a decision has to
be made in the case of a non-trivial merge.  Which branch's unique
commits are on the mainline?[2]  bzr's design decision is that the
*target* of the merge ("merge source into target") is the new
mainline.  This is consistent with the workspace-relative notion of
mainline that bzr was forced to adopt.[3]  In a hierarchical workflow
like the Linux kernel[4] where changes always flow from contributors
"up" a hierarchy of lieutenants (each with their own "local" mainline)
to the master integrator of "the" mainline, the workspace-relative
notion *implemented* by bzr corresponds closely to the *socially
effective* mainline of the project.

Unfortunately, this idealized workflow can only be realized if
integration is fast enough that contributors can resync to "the"
mainline after their "old" work is committed to "the" mainline, but
before committing new work to the local branch.  (Alternatively, they
can commit whenever they like, but use rebase to ensure that their
workspace DAGs are compatible with the mainline DAG.)

What happens in practice is that contributors are impatient to get new
content from "the" mainline (and perhaps from other "more central"
branches).  So they merge "the" mainline back into their local
branches and start work again.  In bzr, this is *fundamentally wrong*.
For Emacs, socially, the Savannah trunk is the mainline, and
contributors think of it that way.  *But bzr doesn't.*  bzr thinks that
this branch labelled "trunk" is just another branch to be merged into
the mainline, *which by design lives in the local branch*.  The social
mainline and the bzr mainline *must* be different!  (Except in the
case of a project run by a benevolent dictator, and even then, they
only coincide for the dictator.)

So here is the primary role of the bound branch in the workflow
recommended by BzrForEmacsDevs.  It always has the socially-determined
mainline as its bzr mainline.  If you commit something there, it
automatically is committed to the trunk repo as well.  If the trunk
repo has advanced, you can't commit in your bound branch.  You need to
update the workspace and remerge your changes, first.  (The remerge is
automatic unless the update creates conflicts.  Note that you still
need to commit your changes.)

You can achieve the same effect by pulling from trunk into your
unbound integration branch (the branch from which you push) before
doing the merge-commit-push dance there.  The bound branch does not do
an implicit update before merge or commit, although it does do an
implicit push after commit.  So the BzrForEmacsDevs bound branch
workflow might save you some typing, and since it always tries to
commit to the master branch first, you can't screw up the mainline.[5]

Footnotes: 
[1]  In git, OTOH, a record is kept of the source of a branch, and for
remote branches a tracking branch is established to represent the
mainline of that branch.  "The" project mainline is conventionally
called "master" (and of course in workspaces has a tracking branch).
In practice, this works well with a bit of discipline for most people.

[2]  In a DAG-based VCS, it doesn't really make sense to build a
mainline out of some commits from one branch and some from the other,
because that requires breaking the parent-child relationships.

[3]  It's not quite true that there was no alternative.  An
alternative would be that commits in a branch are off-mainline, and
the user makes an explicit decision as to which of the branches being
merged is the mainline at the time of the merge.  git actually had a
UI like this (all parents had to be explicitly stated in a merge
command), but it was quickly abandoned.  (Now HEAD is implicitly the
leftmost parent in all git merges).

[4]  As generally understood; in fact it really needs to work quite
differently.

[5]  In the current configuration of the Savannah trunk, you can't
screw up the mainline anyway.  Your push won't be accepted if it would
cause the trunk mainline to be subordinated to your mainline.
However, this is likely to confuse users; the bound branch is easier
to understand.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>
> Date: Fri, 20 Aug 2010 23:41:13 +0100
> 
> >> Sure.  So don't do that.  One (probably for values of "one" != "eliz")
> >> can pull instead, and rebase when necessary to get that to succeed.
> >
> > Please show this workflow in more detail.  Without the details, it's
> > hard to tell what you suggest.  You could be thinking about things
> > possible with git, but not with bzr; it happened in the past.
> 
> I think one needs to do a pull before each commit, to be safe.  (Seems to be 
> the same as update-commit cycle for bound branches.)
> 
> If you have already done a series of commits and lost the opportunity to pull, 
> then you have to rebase.
> 
> If even that seems too far out, then you have to make your changes into a 
> separate branch and merge it in later.

Well, Stephen refused to tell more, citing my imaginary unwillingness
to know.  (Since when asking a question means you don't want to know
the answer?)  So I guess we will never know what Stephen was talking
about.

Btw, pull is only possible when the branch didn't diverge from
upstream, which means it is not suited well for a branch where you do
development.  At least AFAIU.  I'm not even sure it will agree to do
its job if you have uncommitted local changes.

Merging into a local branch is what I had in mind, and the
disadvantage with the "bzr log" output I talked about applies to that
workflow.  To me, it is a significant disadvantage: I use "bzr log" a
lot, and it is painfully slow with the -n0 switch.

As for rebase, I don't really understand what you suggest, and how it
will help with an unbound branch where you develop.  Perhaps you could
elaborate.  (I know what's a rebase and how to do it with Bazaar.)

> However, we get to use our local mirror of the trunk as if it were a local 
> branch.  We can commit to it without affecting anybody else and do a public 
> commit only in the end.  We can commit to it as often as we please, keep a 
> careful log, do enough testing etc. And, this local branch can magically turn 
> back into a proper mirror as soon as we do a rebase-push.  You can't get this 
> with bound branches.  Your local branches have to be necessarily separate 
> branches, whose histories will appear as merges in the mainline.

Sure.  But that's in theory.  In practice, you also need to consider
what kind of changes you make locally.  At least for me, there are 2
main kinds: bug-fixes and development of significant new features.
(There's a third kind as well: when I need to make changes for the
MS-DOS build, but I don't think this part will be interesting for
anyone but myself.)

For the first kind, it is IMO not very good to have the changes
uncommitted upstream for prolonged periods of time, because people are
using the development code, and the bugs annoy them.  So you'd like to
commit those to upstream frequently.  And if you commit frequently,
the proposed rebase-push workflow (IIUC) is not a real improvement,
perhaps even a nuisance, because it adds the overhead of the rebase
and does not save me from frequent slow commits to the master
repository.  And if your local commits have bug-fixes and new features
interspersed, and you want to commit just the bug-fixes, won't the
workflow you need for such cherry-picking become even more
complicated?

For the second kind of changes, I don't see how doing that on an
unbound "mirror" branch is different from a separate branch whose
purpose is development.  A separate branch has the advantage of
keeping the development separate from mainline, so I cannot by mistake
commit unfinished code to upstream.  It also lowers the mental burden
on my mind, by keeping things spatially separated.  Merges are
painless either way; I generally find bzr doing its job very well when
I merge.

So bottom line, I see no advantages to using an unbound branch which
mixes changes of different types.  I agree it's probably possible to
do all that with an unbound branch, but I don't see how it would
relieve the pain of slow commits and slow updates/merges from
upstream, which both are slow due to network traffic bzr incurs when
it uses SFTP.

> We use separate local branches as well, when they are appropriate.  But we are 
> not forced to use them all the time.  (Generally, I am happy to use the main 
> branch for a few days worth of work where I can see the end clearly.  I use 
> separate branches for things that might take a few weeks, or things of a more 
> open-ended nature.)

Well, you see, for me, "a few days worth of work" means in practice a
few weekends of work, because I have almost no time on weekdays for
any significant work.  A week's worth of commits to upstream by others
is most of the time significant enough to make it a PITA when you have
local changes, because there isn't a week that I don't find some small
bug I want to fix and commit almost immediately, to let others enjoy
the fix.

> With bound branches, your branch is locked up until the commit goes
> through.  You can't do anything while you have uncommitted changes
> in your source.

Why would you say that?  That's not true.  Nothing prevents me from
editing while "bzr ci" churns away.  The system is not locked, only
(some) bzr operations will fail.  But most of the development is not
about bzr ops, its about using the editor, the compiler, and other
development tools, none of which are locked up when "bzr ci" runs.

> With unbound branches, we can continue working on the source even
> when push is running in the background, because the source tree
> doesn't have any uncommitted changes.  We can also give up on the
> push if necessary and continue committing to the branch.  The
> advantage seems quite clear to me.

Seems you think that when I'm done with developing one feature or
bug-fix, I immediately proceed to committing the next one.  But that's
not so, at least not here.  Before I'm ready for the next commit,
either locally or to upstream, I need to develop and test it.  And
that part, which is what I do most of the time, is not blocked by a
running commit.

> > No big deal.  But the disadvantages of each workflow need to be well
> > understood before making a decision which one to use.
> 
> I can't believe that you guys really understood the disadvantages of your 
> current workflow before settling on it.

Not entirely, which is why I constantly revise it and make amendments.
But that doesn't mean I shouldn't consider the merits and demerits
beforehand.  After all, most bzr commands are common to all workflows,
and so getting experience with them makes you smarter when making
decisions about workflows you didn't yet try.

> The most illuminating revelation I found in Stephen's post is that,
> with bound branches, every commit is a merge.  Most of the time, it
> is untested and unsafe.  This is the biggest disadvantage of all, if
> you ask me.

I don't understand this argument at all.  In fact, I think it's plain
false.  My workflow in a bound branch is this:

  brz up
  [build the current upstream]
  [some minimal sanity checks to make sure upsteam works]
  [make changes]
  [build and test the modified binary]
  [repeat the sanity checks, fix anything that became broken]
  [bzr up]
  [if there are any changes upstream, build and test again]
  [repeat last two steps until "bzr up" brings no changes]
  bzr ci

Now please tell me how can I commit code that is "untested and unsafe"
with this workflow?  What am I missing?

> It also occurs to me that, by asking everybody to use bound
> branches, you have massively increased the contention on your public
> repo and the server.  That is making your problems worse.

For the record, I didn't write the recommended workflow (I don't even
use it to the letter).  It was written by 2 people who know a lot
about Bazaar and dVCSs.  So I trust them to have weighed the merits
and demerits when they decided on that workflow.

Having said that, the Emacs wiki is free for anyone to suggest
alternative workflows, so if you think there are better alternatives,
feel free to advertise them.  There's already one such alternative
there, I see no reason why there shouldn't be more of them.

Re: Locks on the Bzr repository

[Uday S Reddy]

Eli Zaretskii writes:

> Well, Stephen refused to tell more, citing my imaginary unwillingness
> to know.  (Since when asking a question means you don't want to know
> the answer?)  So I guess we will never know what Stephen was talking
> about.

Well, the strategies I mentioned are common knowledge.  If Stephen has
some other sophisticated tricks he knows about, I am sure he will tell
us.  

> Btw, pull is only possible when the branch didn't diverge from
> upstream, which means it is not suited well for a branch where you do
> development.  At least AFAIU.  I'm not even sure it will agree to do
> its job if you have uncommitted local changes.

Pull works before you commit.  It is the same as update, just a
difference in terminology.

If you commit wihout pulling first, then your branch may begin to
diverge from the public repo.  A bound branch won't allow you to do
this.  It will force you to update first.  So, I can see a reason why
the Stefan and Yidong might recommend bound branches.  But the
reasoning is weak.

If I do 

  commit
  push

and the push fails because the branch has diverged, then my strategy
is to 

  uncommit
  pull
  commit
  push

So, for a single commit, you can get the same effect as bound
branches.

If I do multiple commits to my branch and it has diverged
significantly from the public repo, then I do

  rebase
  ... resolve conflicts
  ... test
  push

(Even though you said you understand how rebase works, here is a quick
summary.  If you originally pulled at revision 500, and made commits
501..505 while the central repo moved up to revision 510 in the
interim, then doing a rebase will import revisions 501..510 from the
central repo and move your commits to 511..515.

This has a significant advantage over bound branches.  I am able to
use my main branch as if it were a task branch, keep all the related
commits together, test all of the commits together, push at my own
convenience, reduce the time wasted in synchronization and ensure that
all the commits are at the top level of the history.  (Bound branches
do the opposite.  They intersperse your commits with others.  You
might end up publishing your commits before they are fully tested.
And, they force you to synchronize with the central repo for each
commit.  In essence, bound branches are not distributed.  They are
centralized.)

> Merging into a local branch is what I had in mind, and the
> disadvantage with the "bzr log" output I talked about applies to that
> workflow.  To me, it is a significant disadvantage: I use "bzr log" a
> lot, and it is painfully slow with the -n0 switch.

I understand.  If you use unbound branches, you don't have to do
this. 

> For the first kind, it is IMO not very good to have the changes
> uncommitted upstream for prolonged periods of time, because people are
> using the development code, and the bugs annoy them.  So you'd like to
> commit those to upstream frequently.  And if you commit frequently,
> the proposed rebase-push workflow (IIUC) is not a real improvement,
> perhaps even a nuisance, because it adds the overhead of the rebase
> and does not save me from frequent slow commits to the master
> repository.  And if your local commits have bug-fixes and new features
> interspersed, and you want to commit just the bug-fixes, won't the
> workflow you need for such cherry-picking become even more
> complicated?

Commiting/sychronizing "frequently" is still better than commiting
instantly.  It gives you a choice as to how frequently you
synchronize.  That might depend on the urgency of the fixes and server
contention levels.  Bound branches remove that choice.

The overhead of rebase is also part of the choice.  In cases where the
overhead is significant, you might push more frequently.  In cases
where it isn't, you might push less frequently.

As for interspersing bug-fixes and new features, I would pick and
choose depending on the situation.  A half-finished new feature
sitting in the trunk doesn't hurt anybody.  I might leave it in.
(People are expected to use the new feature only after it gets into
the NEWS file.)  Or, if I don't feel comfortable about it, I would
probably push my current main branch as a task branch, reset the main
branch to the old state (using uncommit), and attend to the urgent bug
fix in the main branch.  Or, I might do the urgent bug fix in a new
mirror of the central repo and push it.  It will then reappear as part
of rebase in my main branch.  So, there are lots of ways of doing it.

As I said, for big new features, I would use separate task branches. 

> For the second kind of changes, I don't see how doing that on an
> unbound "mirror" branch is different from a separate branch whose
> purpose is development.  A separate branch has the advantage of
> keeping the development separate from mainline, so I cannot by mistake
> commit unfinished code to upstream.  It also lowers the mental burden
> on my mind, by keeping things spatially separated.  Merges are
> painless either way; I generally find bzr doing its job very well when
> I merge.

Agreed.

> So bottom line, I see no advantages to using an unbound branch which
> mixes changes of different types.  I agree it's probably possible to
> do all that with an unbound branch, but I don't see how it would
> relieve the pain of slow commits and slow updates/merges from
> upstream, which both are slow due to network traffic bzr incurs when
> it uses SFTP.

Ok, here, once again are the advantages:

1. Allows you to synchronize with central repo less frequently.

2. Allows you to choose when to synchronize.

3. Keeps your related commits to the mainline together.

4. Allows you to go from bug-fixing mode to (small) development mode
and back, without much pain.

5. Allows you to choose between many different workflows, and even to
switch from one to the other on the fly, whereas bound branches seem
to allow only one workflow.

> Well, you see, for me, "a few days worth of work" means in practice a
> few weekends of work, because I have almost no time on weekdays for
> any significant work.  A week's worth of commits to upstream by others
> is most of the time significant enough to make it a PITA when you have
> local changes, because there isn't a week that I don't find some small
> bug I want to fix and commit almost immediately, to let others enjoy
> the fix.

Ok, I understand.  Most of my work is done during weekends as well.
If I am in a bug fixing mode, I might work on several of them in a
day, but perhaps synchonize only once in the day.  

> Why would you say that?  That's not true.  Nothing prevents me from
> editing while "bzr ci" churns away.  The system is not locked, only
> (some) bzr operations will fail.  But most of the development is not
> about bzr ops, its about using the editor, the compiler, and other
> development tools, none of which are locked up when "bzr ci" runs.

Ok, I stand corrected.  I don't have experience with long-running
commits.  

> Seems you think that when I'm done with developing one feature or
> bug-fix, I immediately proceed to committing the next one.  But that's
> not so, at least not here.  Before I'm ready for the next commit,
> either locally or to upstream, I need to develop and test it.  And
> that part, which is what I do most of the time, is not blocked by a
> running commit.

Agreed.  It is not that the synchronization itself might be blocking
you for further commits (although Jason might be in a situation where
that happens as well).  But, when you want to synchronize, you need to
resolve conflicts which you may not be ready to do at that time.  You
can then postpone synchronization and continue working on other commits.

> I don't understand this argument at all.  In fact, I think it's plain
> false.  My workflow in a bound branch is this:
> 
>   brz up
>   [build the current upstream]
>   [some minimal sanity checks to make sure upsteam works]
>   [make changes]
>   [build and test the modified binary]
>   [repeat the sanity checks, fix anything that became broken]
>   [bzr up]
>   [if there are any changes upstream, build and test again]
>   [repeat last two steps until "bzr up" brings no changes]
>   bzr ci
> 
> Now please tell me how can I commit code that is "untested and unsafe"
> with this workflow?  What am I missing?

I agree that you are being very disciplined here.  But, most of us
think of update/pull/rebase as just a chore to get through before we
can push/commit.  Only if it gives rise to merge conflicts would we
think there is need for further testing.  So, the more
update/pull/rebase we have to do, the more unsafe the code is.  But,
how much this is a problem in reality will probably depend on the
project and the nature of changes being made.  I don't have a lot of
experience here.  But it is a concern, and it got raised for me only
after reading Stephen's message.

> For the record, I didn't write the recommended workflow (I don't even
> use it to the letter).  It was written by 2 people who know a lot
> about Bazaar and dVCSs.  So I trust them to have weighed the merits
> and demerits when they decided on that workflow.

Ok, for the record, I don't mean you by saying "you".  I mean the
emacs-developer team in general.  It seems to me that the closeness to
the previous workflow using CVS was perhaps the overriding criterion
for the authors of the recommended workflow.  The technical merits and
demerits were probably not at the top of the scale.

Having always used unbounded branches, I came into this discussion
wanting to learn why bound branches are being used by you guys.  I
still don't know why.  But I can see that it makes you feel a lot more
comfortable. 

Cheers,
Uday

Re: Locks on the Bzr repository

[Óscar Fuentes]

Uday S Reddy <u.s.reddy@cs.bham.ac.uk> writes:

[snip]

> It seems to me that the closeness to the previous workflow using CVS
> was perhaps the overriding criterion for the authors of the
> recommended workflow.  The technical merits and demerits were probably
> not at the top of the scale.

Not really (for the workflow advertised by Stephen and Karl.)

At the time (and possibly still now) rebasing was not the soundest part
of bzr. So the remaining workflows were "merge+push" and
"pull". "merge+push" have the serious inconveniences described on my
other post on this thread. So Stephen went for the remaining one:
"pull", which is simulated with bound branches. When a developer does
"merge+commit" on his local branch bound to upstream, it is the
equivalent of `pull' on the remote branch.

> Having always used unbounded branches, I came into this discussion
> wanting to learn why bound branches are being used by you guys.  I
> still don't know why.  But I can see that it makes you feel a lot more
> comfortable. 

It is not about confort, but about the lesser evil while working with
bzr.

Re: Locks on the Bzr repository

[Uday S Reddy]

On 8/21/2010 2:45 PM, Óscar Fuentes wrote:

> Not really (for the workflow advertised by Stephen and Karl.)

"Stephen" is Stephen Turnbull?  But he is arguing for unbound branches now.

> At the time (and possibly still now) rebasing was not the soundest part
> of bzr. So the remaining workflows were "merge+push" and
> "pull". "merge+push" have the serious inconveniences described on my
> other post on this thread. So Stephen went for the remaining one:
> "pull", which is simulated with bound branches. When a developer does
> "merge+commit" on his local branch bound to upstream, it is the
> equivalent of `pull' on the remote branch.

Thank you very much.  This makes perfect sense.  If you don't trust rebase, and 
there are good reasons not to, then the current workflow is probably the best 
way of doing things.  And, given the workflow, bound branches serve the purpose 
perfectly fine.  I get it now.

Given the situation, moving to git would seem like a good idea, because its 
rebase is trustworthy and should improve matters enormously.

Cheers,
Uday

Re: Locks on the Bzr repository

[Óscar Fuentes]

Uday S Reddy <u.s.reddy@cs.bham.ac.uk> writes:

>> Not really (for the workflow advertised by Stephen and Karl.)
>
> "Stephen" is Stephen Turnbull?

Yes.

> But he is arguing for unbound branches now.

I'm not sure what he is arguing for, because I didn't follow the
thread.

The workflow he described on the wiki page makes lots of sense and was
great to know before too much people learned the hard way why other
apparently reasonable workflows are flawed.

[snip]

> Given the situation, moving to git would seem like a good idea,
> because its rebase is trustworthy and should improve matters
> enormously.

Bzr was chosen not because technical merits, but because political
reasons (it is a GNU project.) I don't see Emacs moving to Git anytime
soon, no matter how much bzr sucks.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Uday S Reddy writes:
 > On 8/21/2010 2:45 PM, Óscar Fuentes wrote:
 > 
 > > Not really (for the workflow advertised by Stephen and Karl.)
 > "Stephen" is Stephen Turnbull?  But he is arguing for unbound
 > branches now.

Only in that sense that somebody (Jan?) claimed that "push" was
deprecated, and I mistook Eli as supporting that position.  I don't
have a problem with use of a bound branch, although I recommend
against doing work in it.  (This may be idiosyncratic; mostly I like
to ensure I always have a pristine copy of upstream around.)

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sat, 21 Aug 2010 13:31:29 +0100
> From: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>
> Cc: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>,
>     emacs-devel@gnu.org
> 
> Commiting/sychronizing "frequently" is still better than commiting
> instantly.  It gives you a choice as to how frequently you
> synchronize.  That might depend on the urgency of the fixes and server
> contention levels.  Bound branches remove that choice.

Strictly speaking, they don't.  There's still "ci --local".

Anyway, I don't care much about this choice, because I always choose
to sync and commit.  I already explained the reasons.

> As for interspersing bug-fixes and new features, I would pick and
> choose depending on the situation.  A half-finished new feature
> sitting in the trunk doesn't hurt anybody.  I might leave it in.

Really??  I doubt many developers will do the same.  I certainly
won't, not as long as the development code is used by many people.

> (People are expected to use the new feature only after it gets into
> the NEWS file.)

I actually read the ChangeLog files each time I resync.  I won't be
surprised if many others did the same to learn about news.  There's
also the emacs-diffs mailing list which is great for watching new
developments; all commits go there, so NEWS don't count.

> Or, if I don't feel comfortable about it, I would
> probably push my current main branch as a task branch, reset the main
> branch to the old state (using uncommit), and attend to the urgent bug
> fix in the main branch.  Or, I might do the urgent bug fix in a new
> mirror of the central repo and push it.  It will then reappear as part
> of rebase in my main branch.  So, there are lots of ways of doing it.

Yes; and all of them are quite complicated and more error-prone than
the simple ones.  To me that means they should be used only when
necessary (which I do), not as a matter of routine.

> 1. Allows you to synchronize with central repo less frequently.
> 
> 2. Allows you to choose when to synchronize.

These two are the same.

> 3. Keeps your related commits to the mainline together.

This one is unrelated to boundness of the branch.  I can do the same
in a bound branch -- I just need to wait with committing until all the
"related" changes are ready to go.

> 4. Allows you to go from bug-fixing mode to (small) development mode
> and back, without much pain.

Not without some pain.

> 5. Allows you to choose between many different workflows, and even to
> switch from one to the other on the fly, whereas bound branches seem
> to allow only one workflow.

Since I have local unbound branches anyway, this is not an advantage
at all.  Whenever I need this, I simply switch to one of the local
branches.  There are bzr tricks to move all your uncommitted changes
to a different branch, and I use them in that case.

> Having always used unbounded branches, I came into this discussion
> wanting to learn why bound branches are being used by you guys.  I
> still don't know why.  But I can see that it makes you feel a lot more
> comfortable. 

As I wrote elsewhere, the only real advantage is the simplicity of the
workflow.  To me, that is important, as is the fact that simplicity
makes my work less error-prone.  With the little time I have, any
errors on my side could take days to be fixed, and I don't like to
cause long-term breakage.  YMMV.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Uday S Reddy writes:

 > Pull works before you commit.  It is the same as update, just a
 > difference in terminology.

Unfortunately, that's apparently not so.  Not only is there a gross
difference (update will do a merge of incommitted work in the
workspace with the new changes from upstream, pull never merges), but
there are subtle differences: pull involves communication between
branches, and updates the workspace as a side effect, while update
involves communication between a branch and a workspace (and as a side
effect, updates the branch from its master if bound).

 > Ok, for the record, I don't mean you by saying "you".  I mean the
 > emacs-developer team in general.  It seems to me that the closeness
 > to the previous workflow using CVS was perhaps the overriding
 > criterion for the authors of the recommended workflow.  The
 > technical merits and demerits were probably not at the top of the
 > scale.

Simplicity was important.  The option of suggesting a full-bore
distributed workflow was discarded early.  However, the suggested
workflow obviously is quite different from CVS.  A CVS-like workflow
is easy to achieve simply by using a lightweight checkout.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Well, Stephen refused to tell more, citing my imaginary
 > unwillingness to know.  (Since when asking a question means you
 > don't want to know the answer?)

When you consistently ask for details before you have enough
understanding of the theory, or at least the pragmatic constraints.

Anyway, I don't use bzr in anger so my actual workflows aren't going
to be particularly useful.

 > So I guess we will never know what Stephen was talking about.

No, I changed my mind about posting to the thread since Jan mentioned
BzrForEmacsDevs and the recommendation for binding the mirror branch.

 > Btw, pull is only possible when the branch didn't diverge from
 > upstream, which means it is not suited well for a branch where you
 > do development.

D'oh.  You just don't ever want to do development in your mirror
branch (unless it's bound).  The reason this is a problem in bzr (and
to some extent hg) is that branches are heavyweight; you don't want
scads of them being made automatically for internal operations.  So
the user has to make mirror branches explicitly when needed.  git
makes branching extremely lightweight, and therefore need not hesitate
to use mirror ("tracking") branches which are read-only to the user,
and hardly visible.

 > Merging into a local branch is what I had in mind,

Really, I'm coming to believe that bzr is fundamentally unsuited to
radically distributed development (which Emacs approximates, since it
has a fairly large group of committers and no required synchronization
protocol).  Merging into a local branch should not require thought!

 > As for rebase, I don't really understand what you suggest, and how
 > it will help with an unbound branch where you develop.  Perhaps you
 > could elaborate.  (I know what's a rebase and how to do it with
 > Bazaar.)

The workflow should be something like

0.  work in "work", a branch of "mirror" (local) which is in turn a
    branch of "trunk" (on Savannah)
1.  pull from "trunk" into "mirror"
2.  rebase "work" on "mirror"
3.  repeat 0-2 until done
4.  cd to mirror & pull from "work" into "mirror" (this will always succeed!)
5.  push from "mirror" to "trunk"
6.  if 5 succeeds, you're done (for now, go to 0 to start the next change)
7.  else pull --overwrite from "trunk" into "mirror"
8.  goto 2

 > Sure.  But that's in theory.  In practice, you also need to
 > consider what kind of changes you make locally.  At least for me,
 > there are 2 main kinds: bug-fixes and development of significant
 > new features.  (There's a third kind as well: when I need to make
 > changes for the MS-DOS build, but I don't think this part will be
 > interesting for anyone but myself.)

That's interesting in itself for anybody maintaining an optional
feature or different platform, I should think.

The other thing is that you have multiple development branches.  This
is where the rebase-based workflow is attractive to me.  The reason is
that I generally want to have an easy way to diff against the branch
parent (so I don't want to merge into the local branch and have mixed
sequences of trunk and local commits, even if the VCS could
theoretically handle it) but I do want my branch to incorporate the
latest trunk commits.

I guess this could also work with the mirror being a branch bound to
"trunk", but it makes me nervous because of the incompleteness of bzr
documentation; often it's not obvious what checkouts (lightweight or
heavyweight) will do in any given situation.  Also, you could have a
script that will do the rebase step in several related branches with
one pull, avoiding unnecessary network traffic.

 > For the first kind, it is IMO not very good to have the changes
 > uncommitted upstream for prolonged periods of time,

What's your definition of "prolonged"?  I'm thinking in terms of
simply committing all changes locally until the end of the work
session, then doing the merge to mirror and push to upstream in a
batch at the end of the session.

 > because people are using the development code, and the bugs annoy
 > them.  So you'd like to commit those to upstream frequently.  And
 > if you commit frequently, the proposed rebase-push workflow (IIUC)
 > is not a real improvement, perhaps even a nuisance, because it adds
 > the overhead of the rebase and does not save me from frequent slow
 > commits to the master repository.

How frequently do you commit?  How finely do you divide changesets?  I
will often make 5 to 10 commits in a session when I'm working on minor
bugs.  I really would not like it if I had to wait even one minute for
a commit of a typo fix.  (I always separate typo fixes from "real bug"
fixes, and although I do batch typo fixes when I make them one after
another, I don't rebase just so that I can group all typo fixes in a
single commit.)  At least when I push to my launchpad branch of
Mailman, there are many seconds of setup cost, which is only incurred
once per push.  And then I go for coffee or dinner or read mail.  (Of
course, I don't have a race condition since it's a private branch.
But I do have to check for success.)

 > And if your local commits have bug-fixes and new features
 > interspersed, and you want to commit just the bug-fixes, won't the
 > workflow you need for such cherry-picking become even more
 > complicated?

Yes.  I don't do that in bzr.  (It's no big deal in git because of the
interactive rebase feature.)

 > > With bound branches, your branch is locked up until the commit
 > > goes through.  You can't do anything while you have uncommitted
 > > changes in your source.

 > Why would you say that?  That's not true.  Nothing prevents me from
 > editing while "bzr ci" churns away.  The system is not locked, only
 > (some) bzr operations will fail.  But most of the development is
 > not about bzr ops, its about using the editor, the compiler, and
 > other development tools, none of which are locked up when "bzr ci"
 > runs.

No, but if you do a typo fix, commit, then find another typo, 50
minutes (according to the recent report) is a very long time to
wait....  I imagine it would also be a bad idea to continue to work on
any of the files currently being committed, as if the commit fails you
need to disentangle the changes by hand.  bzr is no help because it has
no record of which changes are part of the intended commit, and which
ones aren't.  You could avoid saving your post-bzr-ci work, and use
the difference between the buffer and the file, but that seems a bit
fragile.

 > I don't understand this argument at all.  In fact, I think it's
 > plain false.  My workflow in a bound branch is this:
 > 
 >   brz up
 >   [build the current upstream]
 >   [some minimal sanity checks to make sure upsteam works]
 >   [make changes]
 >   [build and test the modified binary]
 >   [repeat the sanity checks, fix anything that became broken]
 >   [bzr up]
 >   [if there are any changes upstream, build and test again]
 >   [repeat last two steps until "bzr up" brings no changes]
 >   bzr ci
 > Now please tell me how can I commit code that is "untested and
 > unsafe" with this workflow?  What am I missing?

That workflow is safe, because the last thing you do before committing
is test (the very last bzr up is a no-op, of course).  That is what is
required for a safe workflow.  But it's very tempting to not do tests
after bzr up succeeds without conflicts, and experience in many
projects is that it is a common practice to omit testing after a merge.

 > > It also occurs to me that, by asking everybody to use bound
 > > branches, you have massively increased the contention on your
 > > public repo and the server.  That is making your problems worse.

That is definitely not true.  The recommended workflow (at least as
Karl and I wrote it) assumed that "one-commit changes" would be
performed in a separate branch, and merged into the mirror (bound
branch) in batches.  Thus it's equivalent to a pull/rebase/push
workflow for the purpose of hits on the public repo.

It was others who insisted on recommending work *in* the bound branch,
which could increase the burden on the server (but I don't know how
much).  IIRC, however, that is documented in a separate wiki page, and
not as part of the BzrForEmacsDevs recommended workflow.

However, I doubt that's really a big problem.  The real problem is
abysmal administration at Savannah (at least for this purpose), and
maybe bzr's inherent slowness.  I don't understand the stale locks
problem; at least it's not frequently reported on the bazaar list.
(That may be because so few people use dumb servers for the trunk
repository, if it's a bug restricted to dumb servers.)

Re: Locks on the Bzr repository

[Leo]

On 2010-08-21 19:59 +0100, Stephen J. Turnbull wrote:
> D'oh. You just don't ever want to do development in your mirror branch
> (unless it's bound). The reason this is a problem in bzr (and to some
> extent hg) is that branches are heavyweight; you don't want scads of
> them being made automatically for internal operations. So the user has
> to make mirror branches explicitly when needed. git makes branching
> extremely lightweight, and therefore need not hesitate to use mirror
> ("tracking") branches which are read-only to the user, and hardly
> visible.

Branching in git is as fast as:

  echo 7adb0b64823cbdcfbbcac9a9e32c6b435f57d889 > .git/refs/heads/mybranch

basically writing a 40-char string to a file. Because it is so cheap it
completely change many people's workflow. Even for my occasional hacking
of emacs, I have:

* Ferrari
  emacs-server
  emacs-trunk
  git-send-email
  ido
  ido-old
  leo
  mac-port
  master
  processes

It is extremely difficult to lose data in git. So rebase is as safe as
commit. Knowing that I can easily reorganise my private history (it
takes ~ 5 seconds) I also use commit as a way of backup. Using git
really gives you the feeling of using the right tool that is as
intelligent as human being as far as content tracking is concerned.

Leo

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Sat, 21 Aug 2010 20:50:35 +0100
> 
> Branching in git is as fast as:
> 
>   echo 7adb0b64823cbdcfbbcac9a9e32c6b435f57d889 > .git/refs/heads/mybranch
> 
> basically writing a 40-char string to a file.

Are all the branches co-located?  If not, how do you cope with the
bootstrap of a new branch, which takes much longer than writing a
string to a file?

Re: Locks on the Bzr repository

[Leo]

On 2010-08-21 20:59 +0100, Eli Zaretskii wrote:
>> Branching in git is as fast as:
>> 
>>   echo 7adb0b64823cbdcfbbcac9a9e32c6b435f57d889 > .git/refs/heads/mybranch
>> 
>> basically writing a 40-char string to a file.
>
> Are all the branches co-located?  If not, how do you cope with the
> bootstrap of a new branch, which takes much longer than writing a
> string to a file?

There's no bootstrap of a new branch. Branch is a 'symbolic-link' and it
is updated to always point to the most recent commit. You check out a
branch by setting HEAD to that commit contained in
.git/refs/heads/mybranch and if it is a working (non-bare) directory the
contents are updated to the tree object contained in the commit.

1. get the commit id
/Users/Shared/sources/emacs23.2 $ git rev-parse Ferrari
ddb950a3e08e3501c223bea9c1086abce4e13cbb

which is the same as

/Users/Shared/sources/emacs23.2 $ git rev-parse HEAD
ddb950a3e08e3501c223bea9c1086abce4e13cbb

2. show the contents of a commit
/Users/Shared/sources/emacs23.2 $ git cat-file -p ddb950a 
tree d1395168668604b1cb603e32c1d999043add1ece
parent ef9134cc92f8ebb266c750629fac7360ba5b75d2
parent 59549342ff44c4d431f52a3baf5e6a7ae0a88759
parent 7f3be0b47a1fa5cc73d45c3d9ed30107749b80f0
author Leo <sdl.web@gmail.com> 1282321429 +0100
committer Leo <sdl.web@gmail.com> 1282321429 +0100

Merge branches 'git-send-email', 'emacs-server' and 'ido' into Ferrari

3. show the tree object in the commit.
/Users/Shared/sources/emacs23.2 $ git cat-file -p d13951
100644 blob a53a5ca2fa970cff2342b8d33da8f22647297da0	.dir-locals.el
100644 blob 32b44ceb0a3628af22d790e5d3442addbec85e2f	BUGS
100644 blob 94a9ed024d3859793618152ea559a168bbcbb5e2	COPYING
100644 blob c51dced40ef1f656df1228e4905a947730469d7d	ChangeLog
100644 blob 005aac08d87f82248da9ac00091d15bed83eb7cc	INSTALL
100644 blob 8f8367518de9ac12d99c242582455e02093a9801	Makefile.in
100644 blob d41304b2f2c62462e8e41926bf50425340e553cb	README
100644 blob f2eb0262e6029a824a20eeb76791a2d4e6e5a0ef	config.bat
100755 blob e792aac60807bae3d8ef9f48c2a66e56fa4a4477	config.guess
100755 blob 5ecc18b6ada024c4aef5941a5667a3855849e057	config.sub
100755 blob 12b34f54a0049750bb92436ab410b4de6256d11a	configure
100644 blob 156ec2773a322be8bab14b2520a35437c277f5c5	configure.in
040000 tree 5c479520810300c1ab592d1faf953e27eeab32f1	doc
040000 tree de14492ef9724818b455cc93e2a4b065e5332d7c	etc
040000 tree 4a8f090a5d9fb6408e3bffb84ac380e6db8bccd2	info
100755 blob 058b26c82d24335834ad0472109d0cbc1c829084	install-sh
040000 tree 4398aef6800173aa3e602149219e60af1a13e96e	leim
040000 tree 012e4b3be3dc8d63632a6555a93917f113eb2dee	lib-src
040000 tree c56fd64ede1aafd8e0342ff992deee52623a45bc	lisp
040000 tree b318762d0a2386129fb9f8d4fab53513bc2daf9c	lwlib
040000 tree 043be1085f76b65bb7917e59f67e540f27e3d405	m4
040000 tree 65ff8af4fbf924bc4bb8b0074a4fe61cc97fd392	mac
100755 blob a5c600ab37e2bf22b8e0d19df38056ccb66ea9ed	make-dist
100755 blob be98de6be01a1ec467902370b497291cb5d89596	mkinstalldirs
100755 blob ee9e355e3e61d569f9102f4387b108c3cf6dfa04	move-if-change
040000 tree b6b6e11664b383023fea78e202f66da3e57567c3	msdos
040000 tree 66ac1c24a91c5992a967721bec7d967045f03bb9	nextstep
040000 tree 0b835e8cd2c8612be7ba09237d5d1829d5dbf26e	nt
040000 tree 2c6e1bbc971437f32599eff8e301d619ac02fa8e	oldXMenu
040000 tree a57e3c9600ef9fd302d20605a776d63a22089e88	src
100755 blob 26853455e2e0ebd3b5bb3c664a0bd7ad564c6599	update-subdirs
100644 blob 0eb98cc82980c42bb6940d2ab5eab95ac9b5907f	vpath.sed

Leo

Re: Locks on the Bzr repository

[Leo]

On 2010-08-21 21:17 +0100, Leo wrote:
> There's no bootstrap of a new branch. Branch is a 'symbolic-link' and
> it is updated to always point to the most recent commit. You check out
> a branch by setting HEAD to that commit contained in
> .git/refs/heads/mybranch and if it is a working (non-bare) directory
> the contents are updated to the tree object contained in the commit.

The following are two excellent references on git:

1. http://www.gitcasts.com/posts/railsconf-git-talk (video, 1h)
2. http://www.newartisans.com/2008/04/git-from-the-bottom-up.html (pdf)

Leo

Re: Locks on the Bzr repository

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> Are all the branches co-located?  If not, how do you cope with the
Eli> bootstrap of a new branch, which takes much longer than writing a
Eli> string to a file?

There are multiple options for having multiple workspaces sharing a
repository, e.g., "git new-workdir".

Tom

Re: Locks on the Bzr repository

[Leo]

On 2010-08-22 02:41 +0100, Tom Tromey wrote:
> There are multiple options for having multiple workspaces sharing a
> repository, e.g., "git new-workdir".

"git new-workdir"?

Leo

Re: Locks on the Bzr repository

[David De La Harpe Golden]

On 22/08/10 02:47, Leo wrote:
> On 2010-08-22 02:41 +0100, Tom Tromey wrote:
>> There are multiple options for having multiple workspaces sharing a
>> repository, e.g., "git new-workdir".
>
> "git new-workdir"?
>

Maybe the git-new-workdir contrib has been blessed into a command?
Actually, I guess Tom meant the contrib, though I'm not running the 
latest git. If you're using the debian git packages, see
/usr/share/doc/git/contrib/workdir/git-new-workdir
(or see the contrib dir in the git sources).

That's also (straying wildly ontopic) near where the git-bundled git.el
and git-blame.el for emacs can be found, see
/usr/share/doc/git/contrib/emacs/

I try to have my bzr experience close to git, so I use a "shared 
no-trees repository" that I make branches from launchpad and then 
savannah emacs into. I branch locally as I see fit within that shared 
repository.  I then use bzr checkout --lightweight to make working 
tree(s) from those branches, "bzr switch" to switch a working tree 
between them.

Using such a shared repo, doing
bzr branch --no-tree trunk/ blah/
cd ..
bzr checkout --lightweight emacs-repo/blah/ emacs-blah/

takes about 10-15 secs on my system, with most of the time in doing the 
new lightweight checkout construction rather than the branch. "bzr 
switch" allows use of the same dir for different branches (one at a 
given time) in git-like fashion and is a bit faster.

Re: Locks on the Bzr repository

[Andreas Schwab]

David De La Harpe Golden <david@harpegolden.net> writes:

> On 22/08/10 02:47, Leo wrote:
>> On 2010-08-22 02:41 +0100, Tom Tromey wrote:
>>> There are multiple options for having multiple workspaces sharing a
>>> repository, e.g., "git new-workdir".
>>
>> "git new-workdir"?
>>
>
> Maybe the git-new-workdir contrib has been blessed into a command?

The git command will happily execute any git-foo in your path if you
type "git foo".

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: Leo <sdl.web@gmail.com>
 > > Date: Sat, 21 Aug 2010 20:50:35 +0100
 > > 
 > > Branching in git is as fast as:
 > > 
 > >   echo 7adb0b64823cbdcfbbcac9a9e32c6b435f57d889 >
 > > .git/refs/heads/mybranch
 > > 
 > > basically writing a 40-char string to a file.
 > 
 > Are all the branches co-located?

I can't speak for Leo, but in my case all my git branches are
colocated by project.  If I had any upstreams that used git, my
integration branches would have repos separate from my working
branches, but the working branches would all share objects (see
below).  I even tried colocating Emacs and XEmacs, which worked (eg,
for cherry-picking changes) but wasn't worth the effort (because I
didn't update Emacs that often, and then XEmacs decided on hg which
didn't do colocated branches at all well at that time).

 > If not, how do you cope with the bootstrap of a new branch, which
 > takes much longer than writing a string to a file?

It doesn't take very long to create a new branch in a separate
workspace.  It takes the time it takes to unpack a source tarball if
you do it sanely.

# get the original, maintain it as a mirror
# takes me less than 2 minutes IIRC; Linus's branch of kernel < 15 min.
$ git clone ssh://scm.kernel.org/git/git.git pristine

# make a new working branch the least efficient usual way
# creates new .git with metadata by copying (a few kB), symlinks to
# object database (probably many MB, a few files), then checks out
# working tree
$ git clone pristine working
# switch HEAD to branch named "working"
$ git checkout -b working

# make a new working branch very efficiently, but somewhat fragile
# shares the object database
$ export GIT_OBJECT_DIRECTORY=`pwd`/pristine/.git/objects
$ mkdir yowsa
$ cd yowsa
# assume we want to start this branch from "master" (git-ese for "trunk")
$ git checkout -b yowsa master

I just don't hesitate to do these operations in git or hg.  I hesitate
to even think about them in bzr or darcs.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Leo <sdl.web@gmail.com>,
>     emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 15:52:44 +0900
> 
>  > If not, how do you cope with the bootstrap of a new branch, which
>  > takes much longer than writing a string to a file?
> 
> It doesn't take very long to create a new branch in a separate
> workspace.  It takes the time it takes to unpack a source tarball if
> you do it sanely.

I was talking about bootstrapping, not about branch creation.

> I just don't hesitate to do these operations in git or hg.  I hesitate
> to even think about them in bzr or darcs.

I just tried to make a local branch out of the Emacs trunk mirror.  It
took 22 seconds on x86_64 GNU/Linux.  Probably not as fast as git, but
fast enough to be a non-issue.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: "Stephen J. Turnbull" <stephen@xemacs.org>
 > > Cc: Leo <sdl.web@gmail.com>,
 > >     emacs-devel@gnu.org
 > > Date: Sun, 22 Aug 2010 15:52:44 +0900
 > > 
 > >  > If not, how do you cope with the bootstrap of a new branch,
 > >  > which takes much longer than writing a string to a file?
 > > It doesn't take very long to create a new branch in a separate
 > > workspace.  It takes the time it takes to unpack a source tarball
 > > if you do it sanely.
 > 
 > I was talking about bootstrapping, not about branch creation.

Oh, you mean "make bootstrap"?  The answer is (a) don't do it very
often (mostly use colocated branches, even for typos vs. feature
branches), (b) plan well enough that I start the "git clone x y; cd y;
make bootstrap; make check" process before starting something else
that takes 20 minutes or so, and (c) work on XEmacs, where 80% of the
Lisp is not in the main tree (most especially not calc or Gnus!) so a
bootstrap and test can be accomplished in less than 20 minutes even on
a 5-year-old iBook G4.

 > I just tried to make a local branch out of the Emacs trunk mirror.
 > It took 22 seconds on x86_64 GNU/Linux.  Probably not as fast as
 > git, but fast enough to be a non-issue.

That's much better than it used to be, and as you say a non-issue.

Re: Locks on the Bzr repository

[Leo]

On 2010-08-22 08:56 +0100, Eli Zaretskii wrote:
>> It doesn't take very long to create a new branch in a separate
>> workspace. It takes the time it takes to unpack a source tarball if
>> you do it sanely.
>
> I was talking about bootstrapping, not about branch creation.

Sorry I might have missed the point since I didn't follow through the
discussion. Only got interested for 'git'. Are you talking about the
overhead of branch creation? I have no idea what bootstrapping means in
this context.

Leo

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>,  emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 09:57:54 +0100
> 
> On 2010-08-22 08:56 +0100, Eli Zaretskii wrote:
> >> It doesn't take very long to create a new branch in a separate
> >> workspace. It takes the time it takes to unpack a source tarball if
> >> you do it sanely.
> >
> > I was talking about bootstrapping, not about branch creation.
> 
> Sorry I might have missed the point since I didn't follow through the
> discussion. Only got interested for 'git'. Are you talking about the
> overhead of branch creation? I have no idea what bootstrapping means in
> this context.

It means "make bnootstrap" that you need to run in each fresh work
tree before it is usable.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>,
>     emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 03:59:10 +0900
> 
>  > For the first kind, it is IMO not very good to have the changes
>  > uncommitted upstream for prolonged periods of time,
> 
> What's your definition of "prolonged"?

Never thought about quantifying that.  It seems like a moot point
anyway, since there's a request not to lump unrelated changes in the
same commit.  (What would you write as a commit message for such a
commit, anyway?)  So I normally commit a change as soon as it's ready
and tested, and if that's in a bound branch, it goes upstream.  In
practice, this tends to happen every hour or two, when the changes are
quick and simple (for the other kind I do it in a separate branch).

I could use "ci --local", but that would cause my changes appear on a
branch when I finally commit upstream, and require the -n0 switch to
"bzr log", so I try to avoid that.

> I'm thinking in terms of
> simply committing all changes locally until the end of the work
> session, then doing the merge to mirror and push to upstream in a
> batch at the end of the session.

That'd be against the "not lumping together unrelated changes"
request.  Maybe I interpret it too strictly, but it will take Stefan's
or Chong's word to make me change this.

> How frequently do you commit?

Very frequently, see above.

> How finely do you divide changesets?

Each changeset deals with a separate feature or bug-fix.  That is how
I understand the rules.

> I will often make 5 to 10 commits in a session when I'm working on
> minor bugs.

Same here, although it's more close to 5 than to 10.

> I really would not like it if I had to wait even one minute for a
> commit of a typo fix.

Well, you cannot make a changeset of a typo fix with anything else,
unless it's another typo.  The problem is that one rarely finds many
typos in one go, except by luck.  So yes, I need to wait before the
next commit.  I do other things, like reading mail or work on some
other problem in the meantime.

>  > > With bound branches, your branch is locked up until the commit
>  > > goes through.  You can't do anything while you have uncommitted
>  > > changes in your source.
> 
>  > Why would you say that?  That's not true.  Nothing prevents me from
>  > editing while "bzr ci" churns away.  The system is not locked, only
>  > (some) bzr operations will fail.  But most of the development is
>  > not about bzr ops, its about using the editor, the compiler, and
>  > other development tools, none of which are locked up when "bzr ci"
>  > runs.
> 
> No, but if you do a typo fix, commit, then find another typo, 50
> minutes (according to the recent report) is a very long time to
> wait....

Well, 50 minutes _is_ long.  It's more like 3 to 5 here, which is also
long, but barely endurable.

> I imagine it would also be a bad idea to continue to work on
> any of the files currently being committed, as if the commit fails you
> need to disentangle the changes by hand.

Right, you need to refrain from saving.  But if I have a long series
of changes to the same files, it generally means they belong to some
larger feature, so I should do it in a local branch anyway.

> The recommended workflow (at least as Karl and I wrote it) assumed
> that "one-commit changes" would be performed in a separate branch,
> and merged into the mirror (bound branch) in batches.

This would again fly in the face of the "don't lump together..."
request, AFAIU.

Re: Locks on the Bzr repository

[Uday S Reddy]

On 8/21/2010 8:56 PM, Eli Zaretskii wrote:

>> I'm thinking in terms of
>> simply committing all changes locally until the end of the work
>> session, then doing the merge to mirror and push to upstream in a
>> batch at the end of the session.
>
> That'd be against the "not lumping together unrelated changes"
> request.  Maybe I interpret it too strictly, but it will take Stefan's
> or Chong's word to make me change this.

If you use an unbound branch with commit-rebase-push cycle, then commits won't 
be lumped together.  They will appear as independent commits in the central 
repo.  Lumping happens if you use bound branches and try to do local commits.

>> The recommended workflow (at least as Karl and I wrote it) assumed
>> that "one-commit changes" would be performed in a separate branch,
>> and merged into the mirror (bound branch) in batches.
>
> This would again fly in the face of the "don't lump together..."
> request, AFAIU.

Exactly.  There is no upside to bound branches.

You could try the rebase-based workflow that Stephen outlined in his message. 
That won't lump the commits together.

Cheers,
Uday

Re: Locks on the Bzr repository

[Stefan Monnier]

> You could try the rebase-based workflow that Stephen outlined in his
> message. That won't lump the commits together.

The problem with it, is that I've found Bzr's rebase to be unreliable.


        Stefan

Re: Locks on the Bzr repository

[Uday S Reddy]

Stefan Monnier writes:

> The problem with it, is that I've found Bzr's rebase to be unreliable.

Well, what can I say?  A "distributed" VCS without rebase is an
oxymoron.  But there you have it.

If Bazaar is a "gnu project", can FSF lean on them to produce a
proper integrated rebase?

Cheers,
Uday

Re: Locks on the Bzr repository

[Teemu Likonen]

* 2010-08-22 10:03 (+0100), Uday S. Reddy wrote:

> If Bazaar is a "gnu project", can FSF lean on them to produce a proper
> integrated rebase?

Bazaar is a GNU project but more strongly and in practice it's a
Canonical Ltd. project with a couple of paid developers (and few
others). Obviously it's free software (as in the GPL v2) so anyone who
cares about Bazaar can work on improving it. For code contribution there
is some kind of policy for assigning copyrights to Canonical Ltd. but I
don't know the details.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Date: Sun, 22 Aug 2010 10:03:17 +0100
> From: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>
> Cc: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>, emacs-devel@gnu.org
> 
> If Bazaar is a "gnu project", can FSF lean on them to produce a
> proper integrated rebase?

They could try.  However, several of the Bazaar developers think that
rebase is a "bad thing", you can find it in the archives.  So...

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Uday S Reddy writes:

 > If Bazaar is a "gnu project", can FSF lean on them to produce a
 > proper integrated rebase?

Unlikely.  The FSF won't even "lean" on Savannah to properly support
the "official" GNU VCS.

I believe that RMS has suggested that if this is "really" a problem
for (some) Emacs developers, then the (affected) Emacs developers
might want to contribute actively to the Bazaar project.

My impression is that the active Bazaar developers are much less
opposed to a proper rebase than they once were, but Bazaar itself
doesn't need it because of their pqm-based workflow, and the Bazaar
developers most interested in this feature are concentrating on other
work (ie, they're not that interested).  So I think a third-party
contribution would receive proper review and some mentoring (mostly
focused on winkling the patch through the Bazaar contribution process,
but some technical help as well).  That's probably the best way.

Caveat: you probably won't get "integrated" rebase, but having a
robust plugin should be good enough (and fits better with the Bazaar
philosophy, both about VCS, and about good development process).

Re: Locks on the Bzr repository

[Richard Stallman]

    Unlikely.  The FSF won't even "lean" on Savannah to properly support
    the "official" GNU VCS.

You're quite mistaken.  It is not a matter of will, or "leaning".
Savannah is run by few volunteers.  It needs more help.

I asked them two weeks ago to support the bzr protocol.
I got a response, which somehow I failed to see then.
I just saw it now.  Two people said they would work on this.
So I wrote to them.

Would anyone here like to help work on Savannah?

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 23 Aug 2010 00:24:53 -0400
> Cc: u.s.reddy@cs.bham.ac.uk, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Reply-To: rms@gnu.org
> 
> I asked them two weeks ago to support the bzr protocol.
> I got a response, which somehow I failed to see then.
> I just saw it now.  Two people said they would work on this.
> So I wrote to them.
> 
> Would anyone here like to help work on Savannah?

What kind of help is needed?

Re: Locks on the Bzr repository

[Glenn Morris]

Eli Zaretskii wrote:

>> Would anyone here like to help work on Savannah?
>
> What kind of help is needed?

If you're talking about Bazaar specifically, the Savannah thread is
instructive, IMO. Eg:

http://lists.gnu.org/archive/html/savannah-hackers-public/2010-06/msg00011.html

    ...the inital setup, [is] the easy part - what we're looking for
    here is somebody willing to _maintain_ the service, including
    handling user requests.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> Cc: rms@gnu.org,  u.s.reddy@cs.bham.ac.uk,  stephen@xemacs.org,
> 	  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> From: Glenn Morris <rgm@gnu.org>
> Date: Mon, 23 Aug 2010 03:52:57 -0400
> 
> Eli Zaretskii wrote:
> 
> >> Would anyone here like to help work on Savannah?
> >
> > What kind of help is needed?
> 
> If you're talking about Bazaar specifically, the Savannah thread is
> instructive, IMO. Eg:
> 
> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-06/msg00011.html
> 
>     ...the inital setup, [is] the easy part - what we're looking for
>     here is somebody willing to _maintain_ the service, including
>     handling user requests.

Thanks.

However, what exactly constitute the maintenance part seems not to be
explained anywhere.  The only thing I found was a list of task for the
initial setup.  Maybe I just missed it: after all, it's hard to read a
thread that has been dragging on for 2 months.

AFAIU, they are currently looking for volunteers to make the initial
setup happen.  In particular, Karl was near the end of doing that when
he resigned.  Again, I'm not sure I understood it all.

Re: Locks on the Bzr repository

[Richard Stallman]

    AFAIU, they are currently looking for volunteers to make the initial
    setup happen.  In particular, Karl was near the end of doing that when
    he resigned.  Again, I'm not sure I understood it all.

If it isn't clear what they need, how about writing to
savannah-hackers and asking for whatever information you would like?

Re: Locks on the Bzr repository

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>AFAIU, they are currently looking for volunteers to make the initial
>setup happen.  In particular, Karl was near the end of doing that when
>he resigned.  Again, I'm not sure I understood it all.

It wasn't at all clear to me whether I was near the end of the bzr+ssh
setup or not -- that's one of the reasons I had to stop.

I believe this post, and the thread in which it is embedded, contains
all the information I could pass on to someone else picking this up:

http://lists.gnu.org/archive/html/savannah-hackers-public/2010-05/msg00024.html

The start of the thread is here, and contains an rough overview of the
plan:

http://lists.gnu.org/archive/html/savannah-hackers-public/2010-05/msg00001.html

-Karl

Re: Locks on the Bzr repository

[Richard Stallman]

    > Would anyone here like to help work on Savannah?

    What kind of help is needed?

The only specific help I know is needed is this job, making bzr+ssh
work.  However, I am sure a lot of other things are needed.
savannah-hackers@gnu.org could tell you what.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Richard Stallman writes:

 >     Unlikely.  The FSF won't even "lean" on Savannah to properly
 >     support the "official" GNU VCS.
 > 
 > You're quite mistaken.  It is not a matter of will, or "leaning".

I don't care what you call it.  I just think the Emacs project
deserves the benefit of whatever influence it was that caused Emacs
to choose a VCS ill-suited to its needs, against the advice of the
majority of those well-informed about VCS, and against the strong
preferences of most Emacs developers with substantial experience with
distributed VCS.

Re: Locks on the Bzr repository

[Richard Stallman]

    If Bazaar is a "gnu project", can FSF lean on them to produce a
    proper integrated rebase?

I can try -- but first I need a clear and self-contained description
of the problem.  Is the problem solely one of occasional unpredictable
failures?

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 23 Aug 2010 00:24:49 -0400
> Cc: u.s.reddy@cs.bham.ac.uk, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Reply-To: rms@gnu.org
> 
>     If Bazaar is a "gnu project", can FSF lean on them to produce a
>     proper integrated rebase?
> 
> I can try -- but first I need a clear and self-contained description
> of the problem.  Is the problem solely one of occasional unpredictable
> failures?

No.  The problem is that the SFTP protocol we use to access the
repository on Savannah needs a very high network throughput to be
reasonably fast.  When I work on fencepost.gnu.org (a rare situation
these days), it takes between 5 and 30 seconds to either update the
tree from the master repository or commit a set of changes, which is
reasonably fast.  But when I'm doing that from home, where I have a
3.5Mb link, it takes 3 to 5 _minutes_, which is terribly slow for such
a fast link.  The occasional failures due to locks are on top of that;
coupled with the generally very slow operation, it makes the situation
just barely bearable.

Others had even worse experiences.  For example, Jason reported that a
commit of a few lines worth of changes took around 30 minutes.

We are told that switching to the bzr+ssh protocol should make things
much better.  So making this switch is the highest priority now, I
think.  If that is not good enough, either, we could take it up with
the Bazaar maintainers, but as long as we use SFTP, they continue to
claim that the protocol is to blame for the slow operation.

See the on-going discussion re: "Setting up bzr+ssh on Savannah" on
savannah-hackers-public mailing list for similar problems experienced
by other GNU projects that use bzr.  Some of the package maintainers
recently threatened in that discussion to take their repositories
elsewhere or switch to some other VCS, because bzr support on Savannah
seems to be abysmally inadequate.

Re: Locks on the Bzr repository

[Uday S Reddy]

Richard Stallman writes:

> I can try -- but first I need a clear and self-contained description
> of the problem.  Is the problem solely one of occasional unpredictable
> failures?

Well, first of all, Bazaar itself does not have rebase.  Somebody
added a plug-in and this has been distributed with Bazaar.  My version
of Bazaar (2.1.0) came bundled with rebase 0.5.5.  The version number
below 1 doesn't inspire much confidence.  And, it has been reported
that it silently drops merges in the branch that is being rebased.  A
really serious bug, if you ask me.  I don't know what other bugs might
be lurking.  Other people that have tried it can perhaps share their
experience.  There are reports that the Bazaar team doesn't regard
rebase as a good technique at all.  So, it is unlikely that they will
provide any better support for it.

On the other hand, distributed maintenance with a clean management of
history necessarily involves rebase.  If 10 people work on bug fixes
independently, we need to be able to arrange those fixes in some
linear order when the fixes come back.  Bazaar's solution is to
provide merge, which lumps each batch of fixes together into a
subhistory without concern for any logical coherence among the
changes.  Such lumping is discouraged by Emacs development team,
understably, because, if the lumps don't have any logical coherence,
it is as if they don't exist at all.  Whoever is looking through the
history will have to look through every lump to find what he/she is
looking for.

In the absence of rebase, the Emacs developers are forced to
synchronize with the mainline for each commit they want to do.  This
wastes time.  It can also be unreliable in the long run, because
developers tend to think of the synchronization steps as being more or
less automatic rather than logical changes that they really are, which
involve review and testing.

If Bazaar is going to be the favoured VCS for all gnu projects because
of its membership in the gnu family, then it has to think about what
the rest of gnu needs.  I understand that they want to provide safe
solutions that novices can use, but they are also tying the hands of
experienced developers from using the best possible solutions.  It
would be best if they provide rebase and provide sufficient warning,
and leave it to the project managers to use them wisely.

Cheers,
Uday

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Uday S Reddy writes:

 > On the other hand, distributed maintenance with a clean management
 > of history necessarily involves rebase.

This is not true in general.  In particular, the workflow suggested in
BzrForEmacsDevs can produce a "clean history"[1] without rebase,
although it may require remerging branches before pushing because of
the race condition.

Other alternatives include the pull workflow (ie, synchronization by
the maintainers -- you can quibble about whether that's "really
distributed", if you like, and I sympathize, but it is an option for a
project), and an automated patch queue as used by the Bazaar project
itself.

 > Bazaar's solution is to provide merge, which lumps each batch of
 > fixes together into a subhistory without concern for any logical
 > coherence among the changes.

That is not a problem of merge.  All VCSes provide merge.  It is a
problem of the workflow and a matter of taste.  Specifically, whether
you care if even single commits (off mainline) are "covered" by a
merge commit (on mainline).  The Bazaar developers consider this a
natural consequence of distributed development, and have adapted the
log command to normally display only the mainline.  (git and Mercurial
have options to do this, or to display only merge commits, which has a
similar effect.)  There remains a small problem of coming up with an
appropriate log message for the merge commit, but in the specific case
of a single commit, you just copy the commit message into the merge
log message.

Some people consider such a history unsightly, and strongly prefer to
rebase.  But this is a matter of taste.

 > In the absence of rebase, the Emacs developers are forced to
 > synchronize with the mainline for each commit they want to do.

This is only true if you insist that every commit (or every commit on
mainline) in the public repo be tested.  But in that case, rebase
doesn't save you any synchronization compared to non-rebase.

There is are at least two alternatives to rebase.  The first is to use
multiple workspaces, one per task, even trivial tasks.  The second is
to use bzr send for each logical changeset, save the bundles to disk,
and then merge them one after another.

If it is acceptable for the mainline to be temporarily untested, then
the developer can test each commit after merging and pushing.  Then of
course they need to fix any problems discovered in a timely fashion.
An alternative is that only the result of the last merge must pass the
tests; then it's reasonable to test before pushing (although not from
"make bootstrap" probably).

 > It would be best if they provide rebase and provide sufficient
 > warning, and leave it to the project managers to use them wisely.

If a project is going to use rebase, git is *strongly* recommended, as
it not only provides rebase, but several other features for creating
"nice" histories such as filter-branch.

Footnotes: 
[1]  AIUI in Emacs practice a "clean history" means that each
conceptual change, whether it is a single typo fix or integration of
a new garbage collector or internal character coding, is represented
by one commit on the mainline.  A large feature like a new garbage
collector might involve many commits on a branch, however.

Re: Locks on the Bzr repository

[Uday S Reddy]

Stephen J. Turnbull writes:

> This is not true in general.  In particular, the workflow suggested in
> BzrForEmacsDevs can produce a "clean history"[1] without rebase,
> although it may require remerging branches before pushing because of
> the race condition.

Yes, the current workflow does achieve clean history, but at the cost
of removing any gestation period for fixes that the developers might
want.  I am surprised that many emacs developers are ok with it, but
my own experience is that a few days of testing through daily use is
necessary for me to develop confidence in some of my fixes.  I would
feel very exposed if I can't commit fixes to my branch without
publishing them at the same time.

>  > Bazaar's solution is to provide merge, which lumps each batch of
>  > fixes together into a subhistory without concern for any logical
>  > coherence among the changes.
> 
> That is not a problem of merge.  All VCSes provide merge.  

No, I don't mean to say that merge itself is a problem.  But, Bazaar's
recommendation to use merge *instead of* rebase is a problem.  Bazaar
developers do seem to understand that merges doesn't give you a clean
history (at least in Bazaar's way of presenting history), but they
underrate its importance.

I am actually happy with Bazaar's presentation of history.  A
straightline history is good for doing bisection when you need to.
The practice of hiding merge histories from the top-level is also nice
for ignoring unnecessary detail.  

Moreover, I think that even normal merged branches would benefit from
rebasing.  Otherwise, they involve a backwards time travel for the
initial commit, which is confusing during review and even worse when
you have to bisect for a change later.

Maybe I am one of those "users coming from central VCS tools with poor
merge tracking".  But I don't see the new ways being necessarily
superior.  

Cheers,
Uday

PS I am trying to keep this part of the thread focused so that Richard
has a coherent story to deal with.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Uday S Reddy writes:
 > Stephen J. Turnbull writes:
 > 
 > > This is not true in general.  In particular, the workflow
 > > suggested in BzrForEmacsDevs can produce a "clean history"[1]
 > > without rebase, although it may require remerging branches before
 > > pushing because of the race condition.
 > Yes, the current workflow does achieve clean history, but at the
 > cost of removing any gestation period for fixes that the developers
 > might want.

Not at all.  I'm not sure if Eli would do a "one-commit" patch in a
separate workspace, and then merge, but he's said several times now
that he uses multiple workspaces, at least for feature branches.  It's
certainly possible to have a "feature branch" for a one-line patch.
So you put your change in a separate workspace and test it there.  You
merge when you're happy with it.  This is nicer with rebase, I admit,
but it's not impossible without it.

 > Bazaar developers do seem to understand that merges doesn't give
 > you a clean history (at least in Bazaar's way of presenting
 > history), but they underrate its importance.

I don't understand.  Especially since you say you're happy with bzr's
presentation of history.

 > Moreover, I think that even normal merged branches would benefit
 > from rebasing.  Otherwise, they involve a backwards time travel for
 > the initial commit, which is confusing during review and even worse
 > when you have to bisect for a change later.

There are multiple schools of thought about this.  Specifically,
bisecting on

  A1 -- ... -- An
 /               \
0                 1
 \               /
  B1 -- ... -- Bm

vs. bisecting on

0 = A1 -- ... -- An -- B1 -- ... -- Bm = 1

will indeed identify a particular Bi commit as the source of the
problem.  However, many developers feel that understanding that A1 and
B1 were both designed in the context of commit 0 is important for
understanding the evolution of A and B.  This information is lost by
the rebase.  And in fact, it may be that the design of B is cleaner or
more central to future development, so that you really want to fix A,
and leave B alone.  So bisecting may lead to poor selection of fixes.

 > But I don't see the new ways being necessarily superior.

I'm not sure what you mean by "necessarily" superior.  They are not
globally superior; indeed, to take advantage of their features
requires at least minimal investment in learning new patterns (even
for bzr, which takes pains to provide commands to emulate
"traditional" workflows), and taking substantial advantage normally
requires change in workflows.

In the projects I've seen make the change, there is almost always a
division, with the most productive contributors (by patch or LOC
count; of course there are other important measures of productivity)
strongly in favor of more distributed practices and willing to make
some investment, and less productive ones happy with the status quo,
and opposing the "unnecessary" change.

Re: Locks on the Bzr repository

[Lluís]

Stephen J Turnbull writes:

> Uday S Reddy writes:
>> Stephen J. Turnbull writes:
>> 
>> > This is not true in general.  In particular, the workflow
>> > suggested in BzrForEmacsDevs can produce a "clean history"[1]
>> > without rebase, although it may require remerging branches before
>> > pushing because of the race condition.
>> Yes, the current workflow does achieve clean history, but at the
>> cost of removing any gestation period for fixes that the developers
>> might want.

> Not at all.  I'm not sure if Eli would do a "one-commit" patch in a
> separate workspace, and then merge, but he's said several times now
> that he uses multiple workspaces, at least for feature branches.  It's
> certainly possible to have a "feature branch" for a one-line patch.
> So you put your change in a separate workspace and test it there.  You
> merge when you're happy with it.  This is nicer with rebase, I admit,
> but it's not impossible without it.

Just to make sure I understand it. Suppose I'm working on a branch with a fairly
large set of changes and it has been merged back to trunk. After a while a bug
is found on my code, which was not thoroughly tested, or a new relatively minor
functionality is added related to the code on my branch. Should this be
committed on my branch and then followed with a merge to trunk? Or should this
live in a completely new branch that will be merged back to trunk once it's well
tested?

Now that I think of it, I think I start to see the reason for nested history and
the suggestion of not using rebase. Even for the smallest bug fix or feature
addition, a full branch should be created, and every change should be thoroughly
tested in there. Then, trunk would be composed of just merge messages like "add
subsystem Y", "add feature X to subsystem Y", "fix bug N on Y". This would
produce a log in trunk that would very closely resemble the contents of a NEWS
file, but at the expense of a strict branch/test/merge discipline, which is not
bad per-se.

Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: Locks on the Bzr repository

[Lluís]

Lluís  writes:
> Just to make sure I understand it. Suppose I'm working on a branch with a fairly
> large set of changes and it has been merged back to trunk. After a while a bug
> is found on my code, which was not thoroughly tested, or a new relatively minor
> functionality is added related to the code on my branch. Should this be
> committed on my branch and then followed with a merge to trunk? Or should this
> live in a completely new branch that will be merged back to trunk once it's well
> tested?

And now that I think of it. Suppose the case (like emacs) of having a main
development branch where new features and bug fixes as merged, and branches for
tagging minor releases as bugs are fixed on each major version (i.e.,
emacs-23).

If a bug is fixed on the release branch (emacs-23) using branch merges, so that
the history of emacs-23 will read "fix bug N", how can the same history clarity
be maintained in trunk? I mean, both the emacs-23 branch and trunk would like to
benefit from fixing those bugs, but if the release branch is merged into trunk,
the nice messages from emacs-23 are lost once merged into trunk, right? And
merging the new feature/fix branch into both emacs-23 and trunk would provide
the desired history structure outcome, but this is kind of troublesome as the
merge operation must be performed twice... I imagine this is exactly the place
where rebase makes sense (merge new branch into emacs-23, then rebase emacs-23
into trunk), but this should in fact be an operation that is performed only by a
small set of developers.

Am I missing something? Or is this precisely the workflow emacs is supposed to
use?

Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: Locks on the Bzr repository

[Óscar Fuentes]

Lluís <xscript@gmx.net> writes:

[snip]

> If a bug is fixed on the release branch (emacs-23) using branch merges, so that
> the history of emacs-23 will read "fix bug N", how can the same history clarity
> be maintained in trunk? I mean, both the emacs-23 branch and trunk would like to
> benefit from fixing those bugs, but if the release branch is merged into trunk,
> the nice messages from emacs-23 are lost once merged into trunk,
> right?

To see them you'll need the -n0 switch, yes. This is an important case
that breaks the "niceness" of having a privileged arm on the DAG.

> And merging the new feature/fix branch into both emacs-23 and trunk
> would provide the desired history structure outcome, but this is kind
> of troublesome as the merge operation must be performed twice... I
> imagine this is exactly the place where rebase makes sense (merge new
> branch into emacs-23, then rebase emacs-23 into trunk), but this
> should in fact be an operation that is performed only by a small set
> of developers.

Rebasing has the undesirable effect of changing the identity of the
commits, so it is no longer possible to easily test the presence of one
change on several branches ("is commit 388743FA2 present in this
branch?") This is another reason for never rebasing commits that were
released to the public. So no, I don't think that `rebase' would be the
right tool for keeping a nice bzr log on the emacs-23/trunk exchange of
commits.

[snip]

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Lluís writes:

 > If a bug is fixed on the release branch (emacs-23) using branch
 > merges, so that the history of emacs-23 will read "fix bug N", how
 > can the same history clarity be maintained in trunk?

See my previous reply for a brief discussion.

 > I mean, both the emacs-23 branch and trunk would like to benefit
 > from fixing those bugs, but if the release branch is merged into
 > trunk,

No, you essentially never want to do that.  The trunk will diverge
quickly from the release branch.  It has better (but risky) fixes for
some bugs, new features requiring changed APIs, etc.  However, some
changes to the release branch will depend on the old code, and trying
to merge that into the trunk will create One Big Mess.  (This is what
"merge branches" *means*: *all* of the code needs to be merged, not
just the "good" code you want.)

 > the nice messages from emacs-23 are lost once merged into
 > trunk, right?

I don't understand what you mean by "losing the messages".  The
messages are part of the commit object, and come with it in a merge.

 > And merging the new feature/fix branch into both emacs-23 and trunk
 > would provide the desired history structure outcome, but this is
 > kind of troublesome as the merge operation must be performed
 > twice...

This is true, anyway.  The nature of branches is that they diverge,
and the changes that are appropriate diverge, too.

 > I imagine this is exactly the place where rebase makes sense (merge
 > new branch into emacs-23, then rebase emacs-23 into trunk),

Actually, most projects do this in the opposite direction.  Features
and fixes go into trunk, first, typically at developer discretion.
Then the release manager picks ("cherry-picks") the bug fixes
appropriate from the release branch.

Re: Locks on the Bzr repository

[Lluís]

Stephen J Turnbull writes:

> Lluís writes:
>> If a bug is fixed on the release branch (emacs-23) using branch
>> merges, so that the history of emacs-23 will read "fix bug N", how
>> can the same history clarity be maintained in trunk?

> See my previous reply for a brief discussion.

>> I mean, both the emacs-23 branch and trunk would like to benefit
>> from fixing those bugs, but if the release branch is merged into
>> trunk,

> No, you essentially never want to do that.  The trunk will diverge
> quickly from the release branch.  It has better (but risky) fixes for
> some bugs, new features requiring changed APIs, etc.  However, some
> changes to the release branch will depend on the old code, and trying
> to merge that into the trunk will create One Big Mess.  (This is what
> "merge branches" *means*: *all* of the code needs to be merged, not
> just the "good" code you want.)

Ah, you're right.


>> the nice messages from emacs-23 are lost once merged into
>> trunk, right?

> I don't understand what you mean by "losing the messages".  The
> messages are part of the commit object, and come with it in a merge.

I meant that, for example, you merge in a branch for a bugfix into the stable
branch. The log in stable will say something like "fix bug N", although this
might be composed of various commits. Then, you merge stable into trunk, but
merging will no longer show "fix bug N" on the leftmost part of the history in
bazaar, so any changes in stable from last merge into trunk will all be "hidden"
under a single "merge from stable", instead of having now a history on trunk
that shows what exactly has modified in stable and merged into trunk.

Of course, this will not happen with a merge from release into trunk (for the
reasons you explained), but still can happen for any other set of 3 or more
branches; as long as the branch "relation chain" is shorter than 3 levels,
everything's OK, but that's essentially a flat model where every feature/fix is
on its own branch (which is never a release nor a trunk branch) and those
feature/fix branches must be merged multiple times, one into each release/trunk
branch where you want to see that change.

This approach can produce exceptionally nice history logs (in the sense that bzr
hides history messages "inside" a merge), but it seems to me that at the expense
of a higher managing effort.

Of course that effor might already be there on any other approach/workflow, it's
just that this is my first encounter with DVCS.


>> And merging the new feature/fix branch into both emacs-23 and trunk
>> would provide the desired history structure outcome, but this is
>> kind of troublesome as the merge operation must be performed
>> twice...

> This is true, anyway.  The nature of branches is that they diverge,
> and the changes that are appropriate diverge, too.

Then you're in favour of the "flat" model I described above.


>> I imagine this is exactly the place where rebase makes sense (merge
>> new branch into emacs-23, then rebase emacs-23 into trunk),

> Actually, most projects do this in the opposite direction.  Features
> and fixes go into trunk, first, typically at developer discretion.
> Then the release manager picks ("cherry-picks") the bug fixes
> appropriate from the release branch.

Completely right, because of what you pointed out on your second paragraph.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Lluís writes:

 > Just to make sure I understand it. Suppose I'm working on a branch
 > with a fairly large set of changes and it has been merged back to
 > trunk. After a while a bug is found on my code, which was not
 > thoroughly tested, or a new relatively minor functionality is added
 > related to the code on my branch. Should this be committed on my
 > branch and then followed with a merge to trunk? Or should this live
 > in a completely new branch that will be merged back to trunk once
 > it's well tested?

I think it's a matter of taste.  After the first merge to trunk, you
have this history:

0 -- T1 -- ... -- Tn -- 1
 \                     /
  B1 --    ...    -- Bm

You can close the original branch, and create a new one, giving this
history:

0 -- T1 -- ... -- Tn -- 1 -- ... -- Tp -- ... -- Tq -- 2
 \                     /              \               /
  B1 --    ...    -- Bm                `---------- Fix

or you can continue on your branch


0 -- T1 -- ... -- Tn -- 1 -- ... -- Tp -- ... -- Tq -- 2
 \                     /                              /
  B1 --    ...    -- Bm -------------------------- Fix

with very similar effect (bzr knows that B1 ... Bm are already in Tq,
so it will apply effectively apply the patch (Bm -> Fix) to Tq).  The
main difference is that you could have a conflict between Fix and some
patch between 1 and Tp in the "continued feature branch" case, but you
would take the changed code into account in producing Fix for the
"make new branch for fix" case.  In the common case that nobody
touches "your" code between 1 and Tq, these are 100% equivalent.

The other issue is workflow.  If the workflow is like Launchpad or
github, where release managers pull in whole branches from
contributors, it may make sense to use the "continued feature branch"
approach.  Then both "stable" and "devel" branches can pull from your
feature branch, without polluting each other.

In a workflow like Emacs's, where contributions go to trunk and
release manager looks for patches to cherry-pick, the "branch per fix"
approach probably makes more sense.

 > Now that I think of it, I think I start to see the reason for
 > nested history and the suggestion of not using rebase. Even for the
 > smallest bug fix or feature addition, a full branch should be
 > created, and every change should be thoroughly tested in
 > there.

Thoroughly tested, yes, but remember, it is never possible to fully
test in a feature branch (unless the feature branch is the first
continuation applied to the tip of the trunk it branched from).

 > Then, trunk would be composed of just merge messages like "add
 > subsystem Y", "add feature X to subsystem Y", "fix bug N on
 > Y". This would produce a log in trunk that would very closely
 > resemble the contents of a NEWS file, but at the expense of a
 > strict branch/test/merge discipline, which is not bad per-se.

I believe that's very much the way the Bazaar developers think of it.

Re: Locks on the Bzr repository

[Lluís]

Stephen J Turnbull writes:

> Lluís writes:
>> Just to make sure I understand it. Suppose I'm working on a branch
>> with a fairly large set of changes and it has been merged back to
>> trunk. After a while a bug is found on my code, which was not
>> thoroughly tested, or a new relatively minor functionality is added
>> related to the code on my branch. Should this be committed on my
>> branch and then followed with a merge to trunk? Or should this live
>> in a completely new branch that will be merged back to trunk once
>> it's well tested?

> I think it's a matter of taste.  After the first merge to trunk, you
> have this history:

> 0 -- T1 -- ... -- Tn -- 1
>  \                     /
>   B1 --    ...    -- Bm

> You can close the original branch, and create a new one, giving this
> history:

> 0 -- T1 -- ... -- Tn -- 1 -- ... -- Tp -- ... -- Tq -- 2
>  \                     /              \               /
>   B1 --    ...    -- Bm                `---------- Fix

> or you can continue on your branch


> 0 -- T1 -- ... -- Tn -- 1 -- ... -- Tp -- ... -- Tq -- 2
>  \                     /                              /
>   B1 --    ...    -- Bm -------------------------- Fix

Which basically means your history will look like:

0
T1
...
Tn
1: merge feature B
...
Tp
...
Tq
2: fix bug on feature B

no matter which of both approaches you take.

Even more, one should not expect that a feature branch will be indefinitely
available for committing new changes into it; which means that at some point in
time, the branch will have disappeared (been deleted by its creator) and fixes
for such feature will be merged from some other sources.

That is to say that the approach of multiple branches for a single feature is
more likely to happen on projects spanning an arbitrarily large amount of time.


[...]
> The other issue is workflow.  If the workflow is like Launchpad or
> github, where release managers pull in whole branches from
> contributors, it may make sense to use the "continued feature branch"
> approach.  Then both "stable" and "devel" branches can pull from your
> feature branch, without polluting each other.

Which would solve my question about how the same fix in multiple branches could
be shown nicely in the history, but at the expense of having to merge twice,
and having to type twice the same merge message ("merge fix for bug N" or
whatever) if you still want a "coherent" history log between branches.


> In a workflow like Emacs's, where contributions go to trunk and
> release manager looks for patches to cherry-pick, the "branch per fix"
> approach probably makes more sense.

Well, the only difference I see is that on the Emacs' workflow it is supposed
that developers have less means to publish their own branches, so release
managers cherry-pick commits from trunk instead of merging developer's branches
into both trunk and stable.

And I suppose the reason for such approach is to minimize the amount of work
that developers require to work with bazaar instead of CVS.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Tue, 24 Aug 2010 23:10:04 +0900
> Cc: emacs-devel@gnu.org
> 
> In a workflow like Emacs's, where contributions go to trunk and
> release manager looks for patches to cherry-pick, the "branch per fix"
> approach probably makes more sense.

The problem is that a new branch needs to be bootstrapped, which takes
time.  What I do to avoid that is keep the feature branch around even
after the last feature I worked on was merged into trunk and committed
upstream.  Then, when I need to work on another feature in a separate
branch again, I make the existing branch a mirror of the then current
trunk with "bzr pull --overwrite" (after "bzr status" and careful
study of its output!).  Voila! I now have a fresh branch that does not
need a bootstrap.

Re: Locks on the Bzr repository

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> The problem is that a new branch needs to be bootstrapped, which takes
> time.  What I do to avoid that is keep the feature branch around even
> after the last feature I worked on was merged into trunk and committed
> upstream.  Then, when I need to work on another feature in a separate
> branch again, I make the existing branch a mirror of the then current
> trunk with "bzr pull --overwrite" (after "bzr status" and careful
> study of its output!).  Voila! I now have a fresh branch that does not
> need a bootstrap.

For this, local, --no-tree branches combined with `bzr switch' may be
more convenient. That essentially emulates git's colocated branches:

# create the branch where you work:
bzr branch trunk work
cd work, bootstrap
cd ..
# create a bugfix branch
bzr branch --no-tree trunk bugfix#3844
cd work
<hack, hack, hack>
bzr commit
# now you work on another bugfix:
cd .. && bzr branch --no-tree bugfix#3871 && cd work
bzr switch bugfix3871
<hack, hack, hack>
bzr commit
# go back to work on the previous bugfix:
bzr switch bugfix3844
....

Re: Locks on the Bzr repository

[Uday S Reddy]

Lluís <xscript@gmx.net> writes:

> Just to make sure I understand it. Suppose I'm working on a branch
> with a fairly large set of changes and it has been merged back to
> trunk. After a while a bug is found on my code, which was not
> thoroughly tested, or a new relatively minor functionality is added
> related to the code on my branch. Should this be committed on my
> branch and then followed with a merge to trunk? Or should this live
> in a completely new branch that will be merged back to trunk once
> it's well tested?

In Bazaar, it is fine to continue adding to a branch after it has been
merged once.  Next time it is merged, Bazaar can figure out which
revisions are new and merge them appropriately.

> Now that I think of it, I think I start to see the reason for nested
> history and the suggestion of not using rebase. Even for the
> smallest bug fix or feature addition, a full branch should be
> created, and every change should be thoroughly tested in
> there. Then, trunk would be composed of just merge messages like
> "add subsystem Y", "add feature X to subsystem Y", "fix bug N on
> Y". This would produce a log in trunk that would very closely
> resemble the contents of a NEWS file, but at the expense of a strict
> branch/test/merge discipline, which is not bad per-se.

There is no harm in creating task branches for individual bug fixes.
But, that in itself doesn't make one virtuous.  The crucial testing to
be done is *after you merge it into trunk*.  Testing in the task
branch is not enough.  Every time you do pull/update/merge, you need
to test again.  

The workflow recommended for "Quick Fixes" in the Wiki has no logical
problem, even though it avoids task branches.  Its only problem is
that it is slow, especially when used with an sftp server.

Cheers,
Uday

Re: Locks on the Bzr repository

[Lluís]

Uday S Reddy writes:

> Lluís <xscript@gmx.net> writes:
>> Just to make sure I understand it. Suppose I'm working on a branch
>> with a fairly large set of changes and it has been merged back to
>> trunk. After a while a bug is found on my code, which was not
>> thoroughly tested, or a new relatively minor functionality is added
>> related to the code on my branch. Should this be committed on my
>> branch and then followed with a merge to trunk? Or should this live
>> in a completely new branch that will be merged back to trunk once
>> it's well tested?

> In Bazaar, it is fine to continue adding to a branch after it has been
> merged once.  Next time it is merged, Bazaar can figure out which
> revisions are new and merge them appropriately.

Sure, my only concern is on how the history will look like. Suppose I have
commits (f1, f2, f3) implemeting some feature. These are merged into trunk as
revision f. Then I fix a simple bug on the feature (ff11) and merge again as
ff1. Fix another bug with multiple commits (ff21, ff22) and fix another one with
a single commit (ff31). If now I merge, both fixes are merged into trunk, and
supposing they are independent from each other, ideally they should appear as
two different entries on the history in trunk (ff2 and ff3).


>> Now that I think of it, I think I start to see the reason for nested
>> history and the suggestion of not using rebase. Even for the
>> smallest bug fix or feature addition, a full branch should be
>> created, and every change should be thoroughly tested in
>> there. Then, trunk would be composed of just merge messages like
>> "add subsystem Y", "add feature X to subsystem Y", "fix bug N on
>> Y". This would produce a log in trunk that would very closely
>> resemble the contents of a NEWS file, but at the expense of a strict
>> branch/test/merge discipline, which is not bad per-se.

> There is no harm in creating task branches for individual bug fixes.
> But, that in itself doesn't make one virtuous.  The crucial testing to
> be done is *after you merge it into trunk*.  Testing in the task
> branch is not enough.  Every time you do pull/update/merge, you need
> to test again.  

> The workflow recommended for "Quick Fixes" in the Wiki has no logical
> problem, even though it avoids task branches.  Its only problem is
> that it is slow, especially when used with an sftp server.

See above. But basically, we agree as long as you merge on each fix.

Of course, this is all to, at the end, have a history in trunk that is shown as:
   ... f ... ff1 .... ff2 .... ff3 ...

so that history quickly shows at a high level what has been changed just by
looking at the "first level" of the history.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: Locks on the Bzr repository

[Uday S Reddy]

Lluís <xscript@gmx.net> writes:

> Sure, my only concern is on how the history will look like. Suppose
> I have commits (f1, f2, f3) implemeting some feature. These are
> merged into trunk as revision f. Then I fix a simple bug on the
> feature (ff11) and merge again as ff1. Fix another bug with multiple
> commits (ff21, ff22) and fix another one with a single commit
> (ff31). If now I merge, both fixes are merged into trunk, and
> supposing they are independent from each other, ideally they should
> appear as two different entries on the history in trunk (ff2 and
> ff3).

If you want ff2 and ff3 to appear as separate merges in the history,
then you merge them separately.  But they can all sit in the same
feature branch at your end.

In my context, I often want to review the contributions that our
people send in (via Bazaar branches on Launchpad).  There I prefer if
they do all their work of a particular feature in a single branch.
Then I can just pull it into my local copy, review it and merge it in.
If they make a new branch for each fix, it will add additional set-up
cost for me to make a copy of their branch.

I might also mention that it was in this context that I first used
rebase.  Since the contributions are often based on some old revision
in the trunk, which is a pain to go back to, I first rebase my copy of
their branch to the current state of the trunk.  Then I review and
test each commit, before merging it in.  I think this is a great
application of rebase.  Not only is it safe (because rebase is done
before review), it is a lot better than merge which will just plonk
everything together and hope for the best!

> Of course, this is all to, at the end, have a history in trunk that
> is shown as: ... f ... ff1 .... ff2 .... ff3 ...
>
> so that history quickly shows at a high level what has been changed just by
> looking at the "first level" of the history.

Sounds perfect!

Cheers,
Uday

Re: Locks on the Bzr repository

[Uday S Reddy]

Stephen J. Turnbull writes:

> Not at all.  I'm not sure if Eli would do a "one-commit" patch in a
> separate workspace, and then merge, but he's said several times now
> that he uses multiple workspaces, at least for feature branches.  It's
> certainly possible to have a "feature branch" for a one-line patch.
> So you put your change in a separate workspace and test it there.  You
> merge when you're happy with it.  This is nicer with rebase, I admit,
> but it's not impossible without it.

Yes, Eli himself said that he wants simplicity, and he suffers the
delays of immediate push's done by the bound branch as a result of
that.  I think most developers here would rather not open up task
branches for small fixes.  I think 

> There are multiple schools of thought about this.  Specifically,
> bisecting on
> 
>   A1 -- ... -- An
>  /               \
> 0                 1
>  \               /
>   B1 -- ... -- Bm
> 
> vs. bisecting on
> 
> 0 = A1 -- ... -- An -- B1 -- ... -- Bm = 1
> 
> will indeed identify a particular Bi commit as the source of the
> problem.  However, many developers feel that understanding that A1 and
> B1 were both designed in the context of commit 0 is important for
> understanding the evolution of A and B.  

Yes, I am better able to understand this view point now as a result of
this debate.  Note that Bazaar's presentation of the above merge is:

  0 = A1 -- ... -- An .......................1
                    \                       /
                     \ -- B1 -- ... -- Bm -/

and I had orignally missed the idea that B1 should be thought of
really as a successor to 0 = A1.  So, going from An to B1 includes
first a backwards time travel (An -> A1) and going from Bm to 1
includes a fast forward (A1 -> An).  Bazaar's presentation makes it
appear as if B1...Bm have been rebased but, in reality, they are
the original commits that are just being stuck in the middle of the
mainline. 

If one is doing a bisection to find a problem, one would do it first
on the mainline, find that the fault occurs somewhere between An and
1, and then do a second bisection on B1...Bm.  If one is doing this by
hand, it is quite confusing in having to deal with A1 again while
looking through B1...Bm.  I would ask, "A1 works ok, but what is B1
adding that is breaking things?"  But, oops, B1 is doing something
entirely different!

> This information is lost by
> the rebase.  And in fact, it may be that the design of B is cleaner or
> more central to future development, so that you really want to fix A,
> and leave B alone.  So bisecting may lead to poor selection of fixes.

I think the loss of information is a myth.  Information will be lost
only if the VCS deliberately loses it.  There is no reason why the VCS
can't keep track of B1...Bm in terms of patches on A1 as well as
patches on An.  You can then rebase it forwards and backwards as often
as you need to, to get the information you want.

Rethinking the issues now, it is becoming clear to me that there are
two independent, but related, issues:

- Storage: You want to store a history that is accurate so that you
can go back to it when you want to.

- Presentation: You want to produce a rebased history for review and
forensic analysis because it is easier to deal with.  One might want
to go back to the accurate history if the rebased history doesn't
work.

I notice that Stefan proposed essentially this idea in the bazaar
developers list, and Michael Haggerty has a more elaborate
presentation of the issues here:

http://softwareswirl.blogspot.com/2009/04/truce-in-merge-vs-rebase-war.html

Cheers,
Uday

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Uday S Reddy writes:

 > > This information is lost by the rebase.  And in fact, it may be
 > > that the design of B is cleaner or more central to future
 > > development, so that you really want to fix A, and leave B alone.
 > > So bisecting may lead to poor selection of fixes.

 > I think the loss of information is a myth.

Unfortunately, it's not.  YMMV, but some people do value the
information that is lost by rebasing.

I suspect that most Emacs people won't care, though, and will prefer
the more easily reviewed rebased branches.

 > Information will be lost only if the VCS deliberately loses it.
 > There is no reason why the VCS can't keep track of B1...Bm in terms
 > of patches on A1 as well as patches on An.  You can then rebase it
 > forwards and backwards as often as you need to, to get the
 > information you want.

You just invented "Darcs".

 > http://softwareswirl.blogspot.com/2009/04/truce-in-merge-vs-rebase-war.html

Note that Haggerty is also basically trying to invent Darcs, but
doesn't quite get there.  It's also very similar to Robert Collins'
loom plugin for bzr.  The idea of loom is to allow flexible traversal
of such complex DAGs during development so that patches can be applied
to the right "thread" of the loom, then propagated "upward" to the
current version automatically.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Tue, 24 Aug 2010 15:31:52 +0900
> Cc: rms@gnu.org, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
>  > Yes, the current workflow does achieve clean history, but at the
>  > cost of removing any gestation period for fixes that the developers
>  > might want.
> 
> Not at all.  I'm not sure if Eli would do a "one-commit" patch in a
> separate workspace, and then merge, but he's said several times now
> that he uses multiple workspaces, at least for feature branches.

I do use a separate branch for one-off commits, but only if they fix,
or need to be tested on, the DOS port.  The DOS port does not support
out-of-tree build, so I need a separate branch for it (which is mots
of the time kept as a mirror of trunk, with "merge --pull").

Re: Locks on the Bzr repository

[Richard Stallman]

    If Bazaar is going to be the favoured VCS for all gnu projects because
    of its membership in the gnu family, then it has to think about what
    the rest of gnu needs.

I don't know enough to have an opinion about the substance of the
question, but it is clear that you need to argue for this in a Bzr
discussion group.  Discussing it here means, in effect, asking me to
convey it to them -- and using me as middleman for an issue I only
barely understand is not very effective.

Re: Locks on the Bzr repository

[Juanma Barranquero]

On Wed, Aug 25, 2010 at 15:03, Richard Stallman <rms@gnu.org> wrote:

> Discussing it here means, in effect, asking me to
> convey it to them -- and using me as middleman for an issue I only
> barely understand is not very effective.

I think what Udday is trying to say (he'll correct me if I'm wrong) is
that it is a bit unfortunate that you can impel a GNU project to use
Bazaar, but you are unable to motivate Bazaar developers, or indeed
Savannah sysadmins, to better cater to GNU projects' needs.

The consequences of that asymmetry have caused a certain amount of
unhapiness. Not your fault, just the outcome of politics.

    Juanma

Re: Locks on the Bzr repository

[Richard Stallman]

    I think what Udday is trying to say (he'll correct me if I'm wrong) is
    that it is a bit unfortunate that you can impel a GNU project to use
    Bazaar, but you are unable to motivate Bazaar developers, or indeed
    Savannah sysadmins, to better cater to GNU projects' needs.

I agree it is unfortunate, and I keep trying.  It looks like some
progress is being made.  If I get a clear explanation of the need for
rebase in Emacs development, that ought to enable me to push the
Bazaar developers more effectively.

Re: Locks on the Bzr repository

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     I think what Udday is trying to say (he'll correct me if I'm wrong) is
>     that it is a bit unfortunate that you can impel a GNU project to use
>     Bazaar, but you are unable to motivate Bazaar developers, or indeed
>     Savannah sysadmins, to better cater to GNU projects' needs.
>
> I agree it is unfortunate, and I keep trying.  It looks like some
> progress is being made.  If I get a clear explanation of the need for
> rebase in Emacs development, that ought to enable me to push the
> Bazaar developers more effectively.

Getting bzr+ssh working on Savannah is far more important than rebasing.
Maybe you could ask the Bzr developers to get in contact with the
Savannah admins, and try to work things out.

In my estimation, the rebasing issue is currently not as urgent.

Re: Locks on the Bzr repository

[Bernardo Barros]

the connection with http is slower with bazaar/git, right? (sorry, i
did not follow)
that was the reason that caused google to choose mercurial over git.
(besides it has similar svn commands and it is way simpler then git
with the same functionalities)

2010/8/31 Chong Yidong <cyd@stupidchicken.com>:
> Richard Stallman <rms@gnu.org> writes:
>
>>     I think what Udday is trying to say (he'll correct me if I'm wrong) is
>>     that it is a bit unfortunate that you can impel a GNU project to use
>>     Bazaar, but you are unable to motivate Bazaar developers, or indeed
>>     Savannah sysadmins, to better cater to GNU projects' needs.
>>
>> I agree it is unfortunate, and I keep trying.  It looks like some
>> progress is being made.  If I get a clear explanation of the need for
>> rebase in Emacs development, that ought to enable me to push the
>> Bazaar developers more effectively.
>
> Getting bzr+ssh working on Savannah is far more important than rebasing.
> Maybe you could ask the Bzr developers to get in contact with the
> Savannah admins, and try to work things out.
>
> In my estimation, the rebasing issue is currently not as urgent.
>
>

Re: Locks on the Bzr repository

[Miles Bader]

Bernardo Barros <bernardobarros2@gmail.com> writes:
> the connection with http is slower with bazaar/git, right? (sorry, i
> did not follow)

[git has a "fast http" protocol now that is the same speed as the native
protocol, but savannah only currently supports the "slow http" protocol
(plus the fast nativeprotocol).]

-Miles

-- 
Twice, adv. Once too often.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Tue, 31 Aug 2010 20:30:30 -0400
> Cc: Juanma Barranquero <lekktu@gmail.com>, emacs-devel@gnu.org,
> 	monnier@iro.umontreal.ca, u.s.reddy@cs.bham.ac.uk
> 
> Getting bzr+ssh working on Savannah is far more important than rebasing.
> Maybe you could ask the Bzr developers to get in contact with the
> Savannah admins, and try to work things out.
> 
> In my estimation, the rebasing issue is currently not as urgent.

100% agreement.

Re: Locks on the Bzr repository

[Stefan Monnier]

>> Getting bzr+ssh working on Savannah is far more important than rebasing.

Very much so!


        Stefan "who also went through a few half-hour pushes recently,
                for fairly trivial changes"

Re: Locks on the Bzr repository

[Richard Stallman]

    Getting bzr+ssh working on Savannah is far more important than rebasing.
    Maybe you could ask the Bzr developers to get in contact with the
    Savannah admins, and try to work things out.

I don't think this needs any help from the Bzr developers.
What we need is more people helping Savannah.

People said, a week or two ago, that they are working on this.
Would you like to keep in touch with them, to track progress
and remind them it's important?

Meanwhile, since the people who would work on rebase
are disjoint from the people who work on Savannah,
we don't have a choice between them.  If you want rebase,
please write that article, and I can use it to push
for rebase support.

Re: Locks on the Bzr repository

[Bastien]

Richard Stallman <rms@gnu.org> writes:

> I don't think this needs any help from the Bzr developers.
> What we need is more people helping Savannah.

I have searched on the Savannah website (both gnu.org and nongnu.org)
and I was not able to find any page mentionning this need.

Did I miss something?

Having such a page would help a lot translating and forwarding it to
potential volunteers.

-- 
 Bastien

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Bastien <bzg@altern.org>
> Date: Wed, 01 Sep 2010 21:10:26 +0200
> Cc: lekktu@gmail.com, Chong Yidong <cyd@stupidchicken.com>, emacs-devel@gnu.org,
> 	monnier@iro.umontreal.ca, u.s.reddy@cs.bham.ac.uk
> 
> Richard Stallman <rms@gnu.org> writes:
> > I don't think this needs any help from the Bzr developers.
> > What we need is more people helping Savannah.
> 
> I have searched on the Savannah website (both gnu.org and nongnu.org)
> and I was not able to find any page mentionning this need.

See the thread starting here:

  http://lists.gnu.org/archive/html/savannah-hackers-public/2010-05/msg00001.html

This is a very long thread, it was still dragging in August, so you
should look in the archives for all the months since May.

Re: Locks on the Bzr repository

[Bastien]

Eli Zaretskii <eliz@gnu.org> writes:

>> I have searched on the Savannah website (both gnu.org and nongnu.org)
>> and I was not able to find any page mentionning this need.
>
> See the thread starting here:
>
>   http://lists.gnu.org/archive/html/savannah-hackers-public/2010-05/msg00001.html

Thanks for the link, Eli.

I was looking for a more general announcement.  Something like

  "Savannah needs your help.  Savannah is looking for talented sysadmins
  and developers to help maintain blahblah".

I found something on the GNU website:

  http://www.gnu.org/help/help.html

  "Help with Savannah. We are especially looking for technical sysadmin
  volunteers to help with underlying infrastructure support. Volunteers
  to help with pending project submissions are also very welcome.
  Writing a specific tool to ease the license-checking job would be
  nice; for that, a good understanding of free software licensing issues
  is highly desirable and Perl experience is also helpful. Please see
  this general information on how to become a savannah hacker and this
  description of the specific license-checking tool we have in
  mind. Please communicate with us on the savannah-hackers-public
  mailing list."

Maybe adding a line about the need of a secure bzr setup on savannah
would make this need more proeminent. 

-- 
 Bastien

Re: Locks on the Bzr repository

[Richard Stallman]

Since there are already some people working on bzr+ssh for savannah,
I am not sure it is useful to mention that specifically in gnu.org/help.
It could cause more confusion than good.

The crucial questions are, are they working hard on the job,
and do they need more people?

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: eliz@gnu.org, emacs-devel@gnu.org
> Date: Thu, 02 Sep 2010 21:24:26 -0400
> 
> Since there are already some people working on bzr+ssh for savannah,
> I am not sure it is useful to mention that specifically in gnu.org/help.
> It could cause more confusion than good.
> 
> The crucial questions are, are they working hard on the job,
> and do they need more people?

The more important question is: when is it anticipated that bzr+ssh
will be available on Savannah.  We've been waiting for this for 9
months now, with no ETA in sight, and it sounds incredible that the
job is so hard as to require such a long time.  If there are
roadblocks that still prevent this from happening, perhaps someone on
Savannah could spell them out.

Re: Locks on the Bzr repository

[Bastien]

Richard Stallman <rms@gnu.org> writes:

> The crucial questions are, are they working hard on the job,
> and do they need more people?

I subscribed to the savannah-hackers-public mailing list and see
if the Savannah hackers want help and how I can help them get it
from the hackers communities I know.

-- 
 Bastien

Re: Locks on the Bzr repository

[Karl Fogel]

Bastien <bzg@altern.org> writes:
>Richard Stallman <rms@gnu.org> writes:
>> The crucial questions are, are they working hard on the job,
>> and do they need more people?
>
>I subscribed to the savannah-hackers-public mailing list and see
>if the Savannah hackers want help and how I can help them get it
>from the hackers communities I know.

(I think this thread has been posted here before, but for what it's
worth...)

I believe the post below, and the thread in which it is embedded,
contains all the information someone would need to pick this up:

  http://lists.gnu.org/archive/html/savannah-hackers-public/2010-05/msg00024.html

The start of that thread is here, and contains an rough overview of the
plan:

  http://lists.gnu.org/archive/html/savannah-hackers-public/2010-05/msg00001.html

Good luck,
-Karl

Re: Locks on the Bzr repository

[Uday S Reddy]

Richard Stallman writes:

> I don't know enough to have an opinion about the substance of the
> question, but it is clear that you need to argue for this in a Bzr
> discussion group.  Discussing it here means, in effect, asking me to
> convey it to them -- and using me as middleman for an issue I only
> barely understand is not very effective.

Oh, it has been discussed in the Bazaar discussion group.  Look for
"rebase" in this listing:

  https://lists.ubuntu.com/archives/bazaar/2010q1/thread.html

I am sure that these kinds of technical discussions would have
happened myriads of times.  

I think the point where your help is needed is in impressing upon them
that gnu projects like Emacs are having to choose inefficient and
time-consuming workflows because bazaar does not support rebase.  If
rebase were available, Emacs developers can learn to use it correctly,
and use more efficient workflows.  So, they are imposting a cost on
the rest of us.

Cheers,
Uday

Re: Locks on the Bzr repository

[Richard Stallman]

    Oh, it has been discussed in the Bazaar discussion group.

If you are not happy with the outcome, and you have arguments to
present, or you just want the Bazaar developers to know how important
the issue is to you, please participate in the discussion there.
Discussing it where the developers will see it can change their minds.
Discussing it here, in their absence, is mean-spirited.

    I think the point where your help is needed is in impressing upon them
    that gnu projects like Emacs are having to choose inefficient and
    time-consuming workflows because bazaar does not support rebase.

I only barely understand the issue.  A few days ago, someone told me
what rebase means, but I don't understand why it is important.  And
even if I understood the arguments, I would not be able to present
them.

However, if the Emacs maintainers believe that rebase is important for
Emacs maintenance, they can write an explanation of why, including why
the alternatives Bazaar recommends are not adequate for Emacs.  I
could not write that myself, but given it, I could show that to the
Bazaar developers and urge them to pay attention.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: "Stephen J. Turnbull" <stephen@xemacs.org>
 > > Cc: Uday S Reddy <u.s.reddy@cs.bham.ac.uk>,
 > >     emacs-devel@gnu.org
 > > Date: Sun, 22 Aug 2010 03:59:10 +0900
 > > 
 > >  > For the first kind, it is IMO not very good to have the
 > >  > changes uncommitted upstream for prolonged periods of time,
 > > 
 > > What's your definition of "prolonged"?
 >
 > Never thought about quantifying that.  It seems like a moot point
 > anyway, since there's a request not to lump unrelated changes in
 > the same commit.

It's not a moot point because

 > I could use "ci --local", but that would cause my changes appear on
 > a branch when I finally commit upstream, and require the -n0 switch
 > to "bzr log", so I try to avoid that.

isn't a problem if you rebase.  There's a small race (about 10 minutes
for me on XEmacs with a make; make check[1]) where somebody might commit
while you're rebasing, which would cause push/bound-commit to fail,
but you get a fast-forward (commits all on mainline) when the push
succeeds.  And it will usually succeed.

 > > I'm thinking in terms of simply committing all changes locally
 > > until the end of the work session, then doing the merge to mirror
 > > and push to upstream in a batch at the end of the session.
 >
 > That'd be against the "not lumping together unrelated changes"
 > request.

Why do you think that?  Separate commits are separate commits.  They
are recorded as separate commits in upstream when you push.

 > > How frequently do you commit?
 > 
 > Very frequently, see above.

Heh.  Given that for you each commit goes upstream, that's frequent.
But with git I've used workflows that substitute commits for
autosaves.  That's what I think of as "very frequent".

 > > How finely do you divide changesets?
 >
 > Each changeset deals with a separate feature or bug-fix.  That is
 > how I understand the rules.

That's what I thought, and

 > > I will often make 5 to 10 commits in a session when I'm working
 > > on minor bugs.
 > 
 > Same here, although it's more close to 5 than to 10.

so your workflow in this respect is roughly similar to my own (ie, the
rate of mainline commits is the same, although I tend to break up
anything that involves more than five minutes work into multiple
commits on a branch, in both git and hg).  Ten is unusual for me, but
not so unusual (especially when I'm wearing my RM hat) that I can
afford to ignore the possibility that I'll want to do ten commits and
end up spending 30 minutes minimum waiting for the server.  That's
like 20% of my typical session, and if any of those commits really
hangs I could end up wasting half the session.

 > > I really would not like it if I had to wait even one minute for a
 > > commit of a typo fix.

 > Well, you cannot make a changeset of a typo fix with anything else,
 > unless it's another typo.  The problem is that one rarely finds
 > many typos in one go, except by luck.  So yes, I need to wait
 > before the next commit.  I do other things, like reading mail or
 > work on some other problem in the meantime.

Right.  Context switches are expensive.  I'd rather avoid them,
myself.  YMMV.  The "commit+ pull rebase push" workflow means that all
of my waiting is concentrated on the last three operations, and my
train of thought is never interrupted (because that part of the
workflow is a no-brainer).

 > > No, but if you do a typo fix, commit, then find another typo, 50
 > > minutes (according to the recent report) is a very long time to
 > > wait....

 > Well, 50 minutes _is_ long.  It's more like 3 to 5 here, which is
 > also long, but barely endurable.

You have more endurance than I.  I am not willing to put up with that.

 > > The recommended workflow (at least as Karl and I wrote it)
 > > assumed that "one-commit changes" would be performed in a
 > > separate branch, and merged into the mirror (bound branch) in
 > > batches.
 > 
 > This would again fly in the face of the "don't lump together..."
 > request, AFAIU.

Then there's something you don't understand.  What don't you
understand?  That this workflow involves committing each of several
conceptual changes separately, then one push (or merge + commit in a
bound branch) to send them all upstream?  Or that each commit is a
separate transaction, cherrypickable and logged as such, even after a
pull/push/bound-commit to a different repo?  Or something else?

Footnotes: 
[1]  Not doing the equivalent of "make bootstrap" is a little risky
because we don't have proper dependencies for Lisp ... this matters
mostly for changes to exported APIs not exercised by the build and
test process, or to exported macro internals, both of which are pretty
rare.  I consider the level of risk acceptable.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: u.s.reddy@cs.bham.ac.uk,
>     emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 16:36:20 +0900
> 
>  > I could use "ci --local", but that would cause my changes appear on
>  > a branch when I finally commit upstream, and require the -n0 switch
>  > to "bzr log", so I try to avoid that.
> 
> isn't a problem if you rebase.

Stefan says bzr's rebase is unreliable.  I only used rebase in toy
use-cases, so I have no first-hand knowledge.  Perhaps Stefan could
elaborate on the problems he knows about.

>  > > The recommended workflow (at least as Karl and I wrote it)
>  > > assumed that "one-commit changes" would be performed in a
>  > > separate branch, and merged into the mirror (bound branch) in
>  > > batches.
>  > 
>  > This would again fly in the face of the "don't lump together..."
>  > request, AFAIU.
> 
> Then there's something you don't understand.  What don't you
> understand?

I just completely misunderstood the use-case you were describing.
Re-reading it now, I don't see any problems of the "lumping" kind,
indeed.  The only problem is with having the changes on a branch
instead of mainline of the history, unless rebase is used.

There's one other problem with holding back commits to minor problems:
someone else could solve the same problem in the meantime.  Apart of
the obvious social issues ("I was the first to fix it"), there's the
issue of wasted effort.  With changes I make for the DOS port, I
casually leave them in a local commit for days, because I'm sure no
one else will touch them.  But with general-purpose changes, we need a
mechanism to announce "I'm working on this one", otherwise the urge to
commit the results of a job well done is too intense, I think.

If all of this were solved, I could probably wait with commits until
the end of the day, which will make the commit slowness much less of
an issue.

I guess no one seriously considered all these issues because we
believed (and still do) that bzr will eventually become faster on
Savannah.  Perhaps one day we will abandon that hope and turn to
resolving these issues instead.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > isn't a problem if you rebase.
 > 
 > Stefan says bzr's rebase is unreliable.

I know it's limited, but I'm pretty sure it works fine in the simple
case I described.

 > I just completely misunderstood the use-case you were describing.

Ah. OK, no problem.

 > indeed.  The only problem is with having the changes on a branch
 > instead of mainline of the history, unless rebase is used.

Yeah, as I wrote elsewhere, bzr doesn't really give you a good way to
work around that (unless you are in a position to pull contributions
into the project mainline, but that doesn't work for Emacs).

 > There's one other problem with holding back commits to minor
 > problems: someone else could solve the same problem in the
 > meantime.  Apart of the obvious social issues ("I was the first to
 > fix it"), there's the issue of wasted effort.

I don't consider that wasted effort, actually.  The person who gets
there first usually gets the credit, the second person is a perfect
candidate to review. :-)

 > I guess no one seriously considered all these issues

Er, no.  Several of us told you, at greater or lesser length, that
these were going to be issues, and that git was a much better fit for
Emacs.  That advice was completely ignored.

 > because we believed (and still do) that bzr will eventually become
 > faster on Savannah.  Perhaps one day we will abandon that hope and
 > turn to resolving these issues instead.

I don't think you can, without redesigning bzr.  Faster, yes, but this
issues about mainline confusion, no.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: u.s.reddy@cs.bham.ac.uk,
>     emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 18:21:31 +0900
> 
>  > There's one other problem with holding back commits to minor
>  > problems: someone else could solve the same problem in the
>  > meantime.  Apart of the obvious social issues ("I was the first to
>  > fix it"), there's the issue of wasted effort.
> 
> I don't consider that wasted effort, actually.  The person who gets
> there first usually gets the credit, the second person is a perfect
> candidate to review. :-)

Code is seldom reviewed in Emacs, the only exceptions are contributors
who explicitly ask for a review.

And there's the issue of conflicts when you resync next (unless the
committed fix is textually identical to the one in your WT).

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Code is seldom reviewed in Emacs, the only exceptions are
 > contributors who explicitly ask for a review.

Uh....  :-)

 > And there's the issue of conflicts when you resync next (unless the
 > committed fix is textually identical to the one in your WT).

I haven't noticed this being an issue.  It was a massive pain in CVS,
but with git at least you revert the transaction causing conflicts and
either do a reverse cherry-pick or an interactive rebase on your tree.
Then redo the pull/merge/update.  Usually not a big deal, even if it's
buried many commits deep in the DAG.

OTOH, if there is reason to insist on your own patch (either it's
better or it's at least as good and you have additional "important"
patches that depend on your version), then you clone the mainline,
undo the conflicting patch, and merge with your "better" branch.

I can't say this will always work, but it has worked for me on the
occasions I've tried it.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: u.s.reddy@cs.bham.ac.uk,
>     emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 20:14:52 +0900
> 
>  > And there's the issue of conflicts when you resync next (unless the
>  > committed fix is textually identical to the one in your WT).
> 
> I haven't noticed this being an issue.  It was a massive pain in CVS,
> but with git at least you revert the transaction causing conflicts and
> either do a reverse cherry-pick or an interactive rebase on your tree.
> Then redo the pull/merge/update.  Usually not a big deal, even if it's
> buried many commits deep in the DAG.

Resolving such conflicts is not a big deal in bzr as well.  It's just
added overhead, that's all.  People who don't have too much time try
to avoid that.

Re: Locks on the Bzr repository

[Jan Djärv]

Stephen J. Turnbull skrev 2010-08-21 20.59:
> The workflow should be something like
>
> 0.  work in "work", a branch of "mirror" (local) which is in turn a
>      branch of "trunk" (on Savannah)
> 1.  pull from "trunk" into "mirror"
> 2.  rebase "work" on "mirror"
> 3.  repeat 0-2 until done
> 4.  cd to mirror&  pull from "work" into "mirror" (this will always succeed!)
> 5.  push from "mirror" to "trunk"
> 6.  if 5 succeeds, you're done (for now, go to 0 to start the next change)
> 7.  else pull --overwrite from "trunk" into "mirror"
> 8.  goto 2
>

I do something similar, but 5 is a commit on a bound branch.  Is there any 
difference between that and push?

	Jan D.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Jan Djärv writes:
 > 
 > 
 > Stephen J. Turnbull skrev 2010-08-21 20.59:
 > > The workflow should be something like
 > >
 > > 0.  work in "work", a branch of "mirror" (local) which is in turn
 > >     a branch of "trunk" (on Savannah)
 > > 1.  pull from "trunk" into "mirror"
 > > 2.  rebase "work" on "mirror"
 > > 3.  repeat 0-2 until done
 > > 4.  cd to mirror& pull from "work" into "mirror" (this will
 > >     always succeed!)
 > > 5.  push from "mirror" to "trunk"
 > > 6.  if 5 succeeds, you're done (for now, go to 0 to start the
 > >     next change)
 > > 7.  else pull --overwrite from "trunk" into "mirror"
 > > 8.  goto 2
 > 
 > I do something similar, but 5 is a commit on a bound branch.  Is
 > there any difference between that and push?

Speaking very precisely, I think you must be doing something different
from what I think you mean.  If "mirror" is a bound branch, then the
pull in step 4 will operate on "trunk" (ie, the Savannah mainline)
before operating on "mirror".  If the pull succeeds, you're done;
Savannah, "mirror", and "work" are all identical (except for
unversioned "junk").  There should be no need for a commit at that
point.

Since I am not sure exactly what you're doing, you shouldn't rely on
the rest of what I say for yourself.  It works for me, but since I'm
guessing about your workflow, I can't be entirely sure if it should
work for you.

In principle, no, there would be no difference in the case of a
success.  It's the failure modes for commit-while-bound and commit-
then-push that are different.

Specifically, in a bound branch, the pull in step 4 from "work" into
"mirror" can fail if someone has committed *on Savannah* while you
were rebasing.  Or, if you merge, or work in the bound branch, instead
of pull, the commit can fail.  This means that bzr has no record of
the work you are trying to push.  If you don't make any changes while
the commit is in process, then the workspace is your record, and you
can just fix the problem (typically, bzr update "mirror", build and
test, and retry the commit).  If you have made changes, however,
you've got to untangle your mess by hand before updating "mirror".

In the commit-then-push workflow, you have the commit, and you can
then continue to work in that workspace while you're waiting for push
to complete (successfully or not), knowing that Bazaar has a record of
all your work up to the push.  (Typically I will commit several times,
test, and then push.)

Re: Locks on the Bzr repository

[Jan Djärv]

Stephen J. Turnbull skrev 2010-08-22 15.13:
> Jan Djärv writes:
>   >
>   >
>   >  Stephen J. Turnbull skrev 2010-08-21 20.59:
>   >  >  The workflow should be something like
>   >  >
>   >  >  0.  work in "work", a branch of "mirror" (local) which is in turn
>   >  >      a branch of "trunk" (on Savannah)
>   >  >  1.  pull from "trunk" into "mirror"
>   >  >  2.  rebase "work" on "mirror"
>   >  >  3.  repeat 0-2 until done
>   >  >  4.  cd to mirror&  pull from "work" into "mirror" (this will
>   >  >      always succeed!)
>   >  >  5.  push from "mirror" to "trunk"
>   >  >  6.  if 5 succeeds, you're done (for now, go to 0 to start the
>   >  >      next change)
>   >  >  7.  else pull --overwrite from "trunk" into "mirror"
>   >  >  8.  goto 2
>   >
>   >  I do something similar, but 5 is a commit on a bound branch.  Is
>   >  there any difference between that and push?
>
> Speaking very precisely, I think you must be doing something different
> from what I think you mean.  If "mirror" is a bound branch, then the
> pull in step 4 will operate on "trunk" (ie, the Savannah mainline)
> before operating on "mirror".

Ah, 4 would be merge from work to mirror.

Re: Locks on the Bzr repository

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Please show this workflow in more detail.

*sigh*  No, I'll just drop it, since you clearly just don't want to
know.

bzr smart server [was Re: Locks on the Bzr repository]

[Glenn Morris]

Eli Zaretskii wrote:

> Worst of all, no progress is being reported regarding turning on
> bzr+ssh on savannah.  It's been 9 months since work on this was
> supposed to have been started, enough time to deliver a baby, and I
> have no idea when (or if) this will happen.  Could those "in the know"
> please tell what is holding this back?  Given the information about
> the difficulties, maybe someone, whose connection is not so fast and
> whose motivation is therefore high, could help.

I have no inside knowledge, but I like to depress myself by following
the thread on the Savannah mailing list.

http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html

etc

Re: bzr smart server [was Re: Locks on the Bzr repository]

[Lennart Borgman]

On Fri, Aug 20, 2010 at 6:08 PM, Glenn Morris <rgm@gnu.org> wrote:
> Eli Zaretskii wrote:
>
>> Worst of all, no progress is being reported regarding turning on
>> bzr+ssh on savannah.  It's been 9 months since work on this was
>> supposed to have been started, enough time to deliver a baby, and I
>> have no idea when (or if) this will happen.  Could those "in the know"
>> please tell what is holding this back?  Given the information about
>> the difficulties, maybe someone, whose connection is not so fast and
>> whose motivation is therefore high, could help.
>
> I have no inside knowledge, but I like to depress myself by following
> the thread on the Savannah mailing list.
>
> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html


It looks like there are real reasons behind the slow migration to
bzr+ssh according to the link Sylvain gave:

  http://savannah.gnu.org/maintenance/Bzr

Re: bzr smart server [was Re: Locks on the Bzr repository]

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Fri, 20 Aug 2010 18:56:36 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, jan.h.d@swipnet.se, 
> 	Andreas Schwab <schwab@linux-m68k.org>, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> On Fri, Aug 20, 2010 at 6:08 PM, Glenn Morris <rgm@gnu.org> wrote:
> > Eli Zaretskii wrote:
> >
> >> Worst of all, no progress is being reported regarding turning on
> >> bzr+ssh on savannah.  It's been 9 months since work on this was
> >> supposed to have been started, enough time to deliver a baby, and I
> >> have no idea when (or if) this will happen.  Could those "in the know"
> >> please tell what is holding this back?  Given the information about
> >> the difficulties, maybe someone, whose connection is not so fast and
> >> whose motivation is therefore high, could help.
> >
> > I have no inside knowledge, but I like to depress myself by following
> > the thread on the Savannah mailing list.
> >
> > http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
> > http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html
> 
> 
> It looks like there are real reasons behind the slow migration to
> bzr+ssh according to the link Sylvain gave:
> 
>   http://savannah.gnu.org/maintenance/Bzr

No real reason could ever be able to explain a 9-month period during
which it was _not_ done.

Re: bzr smart server [was Re: Locks on the Bzr repository]

[Lennart Borgman]

On Fri, Aug 20, 2010 at 7:11 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Lennart Borgman <lennart.borgman@gmail.com>
>> Date: Fri, 20 Aug 2010 18:56:36 +0200
>> Cc: Eli Zaretskii <eliz@gnu.org>, jan.h.d@swipnet.se,
>>       Andreas Schwab <schwab@linux-m68k.org>, monnier@iro.umontreal.ca, emacs-devel@gnu.org
>>
>> On Fri, Aug 20, 2010 at 6:08 PM, Glenn Morris <rgm@gnu.org> wrote:
>> > Eli Zaretskii wrote:
>> >
>> >> Worst of all, no progress is being reported regarding turning on
>> >> bzr+ssh on savannah.  It's been 9 months since work on this was
>> >> supposed to have been started, enough time to deliver a baby, and I
>> >> have no idea when (or if) this will happen.  Could those "in the know"
>> >> please tell what is holding this back?  Given the information about
>> >> the difficulties, maybe someone, whose connection is not so fast and
>> >> whose motivation is therefore high, could help.
>> >
>> > I have no inside knowledge, but I like to depress myself by following
>> > the thread on the Savannah mailing list.
>> >
>> > http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
>> > http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html
>>
>>
>> It looks like there are real reasons behind the slow migration to
>> bzr+ssh according to the link Sylvain gave:
>>
>>   http://savannah.gnu.org/maintenance/Bzr
>
> No real reason could ever be able to explain a 9-month period during
> which it was _not_ done.


If no real reason can explain it what do you believe in then?

Re: bzr smart server [was Re: Locks on the Bzr repository]

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Fri, 20 Aug 2010 19:13:59 +0200
> Cc: rgm@gnu.org, jan.h.d@swipnet.se, schwab@linux-m68k.org, 
> 	monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> >> It looks like there are real reasons behind the slow migration to
> >> bzr+ssh according to the link Sylvain gave:
> >>
> >>   http://savannah.gnu.org/maintenance/Bzr
> >
> > No real reason could ever be able to explain a 9-month period during
> > which it was _not_ done.
> 
> 
> If no real reason can explain it what do you believe in then?

Inaction and lack of motivation.

Re: bzr smart server

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lennart Borgman <lennart.borgman@gmail.com>
>> Date: Fri, 20 Aug 2010 19:13:59 +0200
>> Cc: rgm@gnu.org, jan.h.d@swipnet.se, schwab@linux-m68k.org, 
>> 	monnier@iro.umontreal.ca, emacs-devel@gnu.org
>> 
>> >> It looks like there are real reasons behind the slow migration to
>> >> bzr+ssh according to the link Sylvain gave:
>> >>
>> >>   http://savannah.gnu.org/maintenance/Bzr
>> >
>> > No real reason could ever be able to explain a 9-month period during
>> > which it was _not_ done.
>> 
>> 
>> If no real reason can explain it what do you believe in then?
>
> Inaction and lack of motivation.

That's a very real reason in software development.  One of the most
important reasons that tools should be fun to work with.

-- 
David Kastrup

Re: bzr smart server [was Re: Locks on the Bzr repository]

[Eli Zaretskii]

> Cc: Andreas Schwab <schwab@linux-m68k.org>,  jan.h.d@swipnet.se,
> 	  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 20 Aug 2010 12:08:09 -0400
> 
> Eli Zaretskii wrote:
> 
> > Worst of all, no progress is being reported regarding turning on
> > bzr+ssh on savannah.  It's been 9 months since work on this was
> > supposed to have been started, enough time to deliver a baby, and I
> > have no idea when (or if) this will happen.  Could those "in the know"
> > please tell what is holding this back?  Given the information about
> > the difficulties, maybe someone, whose connection is not so fast and
> > whose motivation is therefore high, could help.
> 
> I have no inside knowledge, but I like to depress myself by following
> the thread on the Savannah mailing list.
> 
> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html

Richard, are you reading this?  Can you _please_ help??

Re: bzr smart server

[Stefan Monnier]

>> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
>> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html

> Richard, are you reading this?  Can you _please_ help??

Moving to Git does seem like a good option ;-)


        Stefan

Re: bzr smart server

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Glenn Morris <rgm@gnu.org>,  Richard Stallman <rms@gnu.org>,  emacs-devel@gnu.org
> Date: Sat, 21 Aug 2010 01:02:15 +0200
> 
> >> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-07/msg00008.html
> >> http://lists.gnu.org/archive/html/savannah-hackers-public/2010-08/msg00001.html
> 
> > Richard, are you reading this?  Can you _please_ help??
> 
> Moving to Git does seem like a good option ;-)

Not for me.  Git is a mess on Windows.

Re: bzr smart server

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Moving to Git does seem like a good option ;-)
>
> Not for me.  Git is a mess on Windows.

It used to be and that was the reason I didn't argue against bzr at the
time. Started to use bzr for my projects just because it treats Windows
as a first class citizen, become frustrated, tried MSYSGit again, it
worked, and ditched bzr for good. Now Git is what manages all projects I
work with, including Emacs, on GNU/Linux and Windows.

Re: bzr smart server

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sat, 21 Aug 2010 15:22:39 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Moving to Git does seem like a good option ;-)
> >
> > Not for me.  Git is a mess on Windows.
> 
> It used to be and that was the reason I didn't argue against bzr at the
> time. Started to use bzr for my projects just because it treats Windows
> as a first class citizen, become frustrated, tried MSYSGit again, it
> worked, and ditched bzr for good. Now Git is what manages all projects I
> work with, including Emacs, on GNU/Linux and Windows.

I don't want to install MSYS.

Re: bzr smart server

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> It used to be and that was the reason I didn't argue against bzr at the
>> time. Started to use bzr for my projects just because it treats Windows
>> as a first class citizen, become frustrated, tried MSYSGit again, it
>> worked, and ditched bzr for good. Now Git is what manages all projects I
>> work with, including Emacs, on GNU/Linux and Windows.
>
> I don't want to install MSYS.

I tend to avoid MSYS/Cygwin too, but the installer of MSYS-Git has
options for making MSYS invisible to the rest of the system. In essence,
it is out of the PATH. So the fact that Git is working on top of MSYS is
an internal implementation detail. It is not different from any other
self-contained application.

And you don't need to separately install MSYS for using MSYS-Git.

Re: bzr smart server

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sat, 21 Aug 2010 19:09:58 +0200
> 
> I tend to avoid MSYS/Cygwin too, but the installer of MSYS-Git has
> options for making MSYS invisible to the rest of the system. In
> essence, it is out of the PATH.

As I'm sure you know too well, Windows looks for programs not only on
PATH.

But I'm uncomfortable with this thread discussing my personal
preferences too much, so let's just drop this.

Re: bzr smart server

[Miles Bader]

Eli Zaretskii <eliz@gnu.org> writes:
>> Moving to Git does seem like a good option ;-)
>
> Not for me.  Git is a mess on Windows.

I think that's not really true any more.

-Miles

-- 
[|nurgle|]  ddt- demonic? so quake will have an evil kinda setting? one that
            will  make every christian in the world foamm at the mouth?
[iddt]      nurg, that's the goal

Re: bzr smart server

[Eli Zaretskii]

> From: Miles Bader <miles@gnu.org>
> Date: Sat, 21 Aug 2010 22:46:04 +0900
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >> Moving to Git does seem like a good option ;-)
> >
> > Not for me.  Git is a mess on Windows.
> 
> I think that's not really true any more.

It is for me, because I won't install MSYS, in any incarnation.

Re: bzr smart server

[Miles Bader]

Eli Zaretskii <eliz@gnu.org> writes:
>> >> Moving to Git does seem like a good option ;-)
>> >
>> > Not for me.  Git is a mess on Windows.
>> 
>> I think that's not really true any more.
>
> It is for me, because I won't install MSYS, in any incarnation.

Why?

[I've no doubt msys is a bit bloated, but OTOH, so is python, and bzr
requires that...]

-miles

-- 
The car has become... an article of dress without which we feel uncertain,
unclad, and incomplete.  [Marshall McLuhan, Understanding Media, 1964]

Re: bzr smart server

[Eli Zaretskii]

> From: Miles Bader <miles@gnu.org>
> Date: Sun, 22 Aug 2010 00:45:41 +0900
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >> >> Moving to Git does seem like a good option ;-)
> >> >
> >> > Not for me.  Git is a mess on Windows.
> >> 
> >> I think that's not really true any more.
> >
> > It is for me, because I won't install MSYS, in any incarnation.
> 
> Why?

Because it's an incompatible environment.  MSYS is a fork of Cygwin,
so it is as incompatible with native Windows development environment,
such as MinGW, as is Cygwin.

Of course, given enough tinkering and management overhead, one can
overcome this difficulty.  But I will resist as much as I can to be
forced into that overhead.

Re: bzr smart server

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Because it's an incompatible environment.  MSYS is a fork of Cygwin,
> so it is as incompatible with native Windows development environment,
> such as MinGW, as is Cygwin.
>
> Of course, given enough tinkering and management overhead, one can
> overcome this difficulty.  But I will resist as much as I can to be
> forced into that overhead.

Someone else already did that. At install time, it is possible
(actually, the default option IIRC) to configure MSYS-Git for staying
out of the way.

Re: bzr smart server

[Jason Rumney]

  On 21/8/2010 11:51 PM, Eli Zaretskii wrote:
> Because it's an incompatible environment.  MSYS is a fork of Cygwin,
It is? I was under the impression that MSYS was an extension of MinGW to 
give a Posix shell and environment in which to use the MinGW tools.

Re: bzr smart server

[Eli Zaretskii]

> Date: Sun, 22 Aug 2010 01:28:14 +0800
> From: Jason Rumney <jasonr@gnu.org>
> 
>   On 21/8/2010 11:51 PM, Eli Zaretskii wrote:
> > Because it's an incompatible environment.  MSYS is a fork of Cygwin,
> It is? I was under the impression that MSYS was an extension of MinGW to 
> give a Posix shell and environment in which to use the MinGW tools.

No.  MSYS is an environment to _build_ MinGW ports (as opposed to
_use_ them).  Its creator forked it from Cygwin for reasons I forgot.
His recommendation is to use MSYS only for building MinGW ports.

Re: bzr smart server

[Óscar Fuentes]

Jason Rumney <jasonr@gnu.org> writes:

>  On 21/8/2010 11:51 PM, Eli Zaretskii wrote:
>> Because it's an incompatible environment.  MSYS is a fork of Cygwin,
>
> It is? I was under the impression that MSYS was an extension of MinGW
> to give a Posix shell and environment in which to use the MinGW tools.

MSYS is a fork of Cygwin. It can't be an extension of MinGW because
MinGW is simply a Windows native port of gcc + binutils (then came gdb,
gmake, etc, but they are real Windows ports, without relying on any
Posix emulation layer.)

The original goal of MSYS was to provide a better environment to run
configure scripts using MinGW as the target. Previously it was done with
Cygwin, but problems with pathnames were frequent. The main feature of
MSYS over Cygwin (at least until recently) was that MSYS understand
Windows pathnames, and can translate them depending on the kind of
executable to call: Posix-style for those executables that use the Posix
emulator, and Windows-style for the rest. That was not 100% reliable,
but usually did a better job than Cygwin.

Re: bzr smart server

[Eric Hanchrow]

On Sat, Aug 21, 2010 at 1:05 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> Not for me.  Git is a mess on Windows.

I recommend that you give git on windows another try -- I've been
using it at work for many months, and it's fine.

Re: bzr smart server

[Lennart Borgman]

On Sat, Aug 21, 2010 at 4:00 PM, Eric Hanchrow <eric.hanchrow@gmail.com> wrote:
> On Sat, Aug 21, 2010 at 1:05 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> Not for me.  Git is a mess on Windows.
>
> I recommend that you give git on windows another try -- I've been
> using it at work for many months, and it's fine.

Is there a git for w32 now or do you still need msys/cygwin?

Re: bzr smart server

[Óscar Fuentes]

Lennart Borgman <lennart.borgman@gmail.com> writes:

>> I recommend that you give git on windows another try -- I've been
>> using it at work for many months, and it's fine.
>
> Is there a git for w32 now or do you still need msys/cygwin?

MSYS-Git is what you need. MSYS is not an issue, as there is no
requirement for putting the executables on the PATH.

Re: bzr smart server

[Lennart Borgman]

On Sat, Aug 21, 2010 at 4:24 PM, Óscar Fuentes <ofv@wanadoo.es> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>>> I recommend that you give git on windows another try -- I've been
>>> using it at work for many months, and it's fine.
>>
>> Is there a git for w32 now or do you still need msys/cygwin?
>
> MSYS-Git is what you need. MSYS is not an issue, as there is no
> requirement for putting the executables on the PATH.


And it works fine from within Emacs too?

Re: bzr smart server

[Óscar Fuentes]

Lennart Borgman <lennart.borgman@gmail.com> writes:

>> MSYS-Git is what you need. MSYS is not an issue, as there is no
>> requirement for putting the executables on the PATH.
>
> And it works fine from within Emacs too?

Yes.

I use magit.el and Emacs' own VC. magit.el already has a variable for
pointing to the git executable and I added one to VC for the same
purpose. So neither MSYS nor git.exe are on the PATH.

Re: bzr smart server

[Richard Stallman]

    > Richard, are you reading this?  Can you _please_ help??

I have not read about this.  Stefan, could you send me a concise
statement of the issue?

Re: Locks on the Bzr repository

[Miles Bader]

Jan Djärv <jan.h.d@swipnet.se> writes:
> It is no fun fixing bugs and adding features and the just keep it to yourself.
> Push has been discuraged on this list and other places several times.

It has...?

I'm thorougly confused...

Isn't that like a basic part of any modern VCS...?

-Miles

-- 
Logic, n. The art of thinking and reasoning in strict accordance with the
limitations and incapacities of the human misunderstanding.

Re: Locks on the Bzr repository

[Jason Rumney]

On 20/08/2010 12:44, Miles Bader wrote:
> Jan Djärv<jan.h.d@swipnet.se>  writes:
>    
>> It is no fun fixing bugs and adding features and the just keep it to yourself.
>> Push has been discuraged on this list and other places several times.
>>      
> It has...?
>
> I'm thorougly confused...
>
> Isn't that like a basic part of any modern VCS...?
>    

Whether it is dicouraged or not, it doesn't solve the problem of bzr 
locks being left behind after network problems and other issues, nor 
does it solve the problem of sftp taking too long to commit/push small 
changes.

Re: Locks on the Bzr repository

[Jan Djärv]

Miles Bader skrev 2010-08-20 06.44:
> Jan Djärv<jan.h.d@swipnet.se>  writes:
>> It is no fun fixing bugs and adding features and the just keep it to yourself.
>> Push has been discuraged on this list and other places several times.
>
> It has...?
>
> I'm thorougly confused...
>
> Isn't that like a basic part of any modern VCS...?
>

It can be modern but still full of bugs and with performance problems.

	Jan D.

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Thu, 19 Aug 2010 17:52:53 +0200
> Cc: emacs-devel@gnu.org
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > If that's indeed the culprit, I think it's a misfeature of Bzr, but at
> > least we can workaround it by using "bzr commit -m <...>", which I would
> > then *strongly* recommend.
> 
> Even better: use an unbound branch.

How would this help?  No matter what you do in an unbound branch, you
eventually need to push or merge, at which time you will have the same
problem.  What am I missing?

Re: Locks on the Bzr repository

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Date: Thu, 19 Aug 2010 17:52:53 +0200
>> Cc: emacs-devel@gnu.org
>> 
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> 
>> > If that's indeed the culprit, I think it's a misfeature of Bzr, but at
>> > least we can workaround it by using "bzr commit -m <...>", which I would
>> > then *strongly* recommend.
>> 
>> Even better: use an unbound branch.
>
> How would this help?  No matter what you do in an unbound branch, you
> eventually need to push or merge, at which time you will have the same
> problem.  What am I missing?

It won't improve the actual transfer of the commits, but it lets you
review before you push.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Locks on the Bzr repository

[Jason Rumney]

  On 19/8/2010 11:40 PM, Stefan Monnier wrote:
> A couple times now, I've had the following kind of message when trying
> to commit:
>
> Unable to obtain lock sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/.bzr/branch/lock
> held by jasonr@gnu.org on host wanchan [process #23356]
> locked 28 minutes, 51 seconds ago
>
> The first time I thought it was a stale lock so I broke it, but then it
> came right back, so I suspect it might be due to something else.
> Could it be that bzr grabs the lock when you do "bzr commit" and keep it
> held for as long as you're editing the commit message?

I don't think editing the commit message is the problem, but the fact 
that to do anything with the master repository involves downloading 
and/or uploading at least 4MB of data, on a connection which fluctuates 
between 0 and 100kb/s, mostly in the 6 - 20kb/s range.

> If that's indeed the culprit, I think it's a misfeature of Bzr, but at
> least we can workaround it by using "bzr commit -m<...>", which I would
> then *strongly* recommend.

I think the misfeature is that we are still using sftp access, which 
might be OK for users in well connected parts of the world like US, 
Europe and Japan, but from here is unusably slow.

Re: Locks on the Bzr repository

[Dan Nicolaescu]

Jason Rumney <jasonr@gnu.org> writes:

>  On 19/8/2010 11:40 PM, Stefan Monnier wrote:
>> A couple times now, I've had the following kind of message when trying
>> to commit:
>>
>> Unable to obtain lock sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/.bzr/branch/lock
>> held by jasonr@gnu.org on host wanchan [process #23356]
>> locked 28 minutes, 51 seconds ago
>>
>> The first time I thought it was a stale lock so I broke it, but then it
>> came right back, so I suspect it might be due to something else.
>> Could it be that bzr grabs the lock when you do "bzr commit" and keep it
>> held for as long as you're editing the commit message?
>
> I don't think editing the commit message is the problem, but the fact
> that to do anything with the master repository involves downloading
> and/or uploading at least 4MB of data, on a connection which
> fluctuates between 0 and 100kb/s, mostly in the 6 - 20kb/s range.
>
>> If that's indeed the culprit, I think it's a misfeature of Bzr, but at
>> least we can workaround it by using "bzr commit -m<...>", which I would
>> then *strongly* recommend.
>
> I think the misfeature is that we are still using sftp access, which
> might be OK for users in well connected parts of the world like US,
> Europe and Japan, but from here is unusably slow.

I have seen the lock problem happen when using -m<...> and on very
well connected machine, so it's might be just bzr being bzr.

Re: Locks on the Bzr repository

[Jason Rumney]

  On 20/8/2010 12:43 AM, Dan Nicolaescu wrote:
> I have seen the lock problem happen when using -m<...>  and on very
> well connected machine, so it's might be just bzr being bzr.

When you say you've seen the problem, is it your own locks you are 
seeing, or locks held by someone else?  Normally you would only see 
other peoples' locks unless you are trying to multitask on the same 
tree, or an operation got interrupted and the lock really is stale.

Re: Locks on the Bzr repository

[Dan Nicolaescu]

Jason Rumney <jasonr@gnu.org> writes:

>  On 20/8/2010 12:43 AM, Dan Nicolaescu wrote:
>> I have seen the lock problem happen when using -m<...>  and on very
>> well connected machine, so it's might be just bzr being bzr.
>
> When you say you've seen the problem, is it your own locks you are
> seeing, or locks held by someone else?  Normally you would only see

My own locks.
Sometimes commits fail (in a up to date tree!) after a long time, and
then I have to remove my own locks to try again.

And it's not even that seldom:

$ grep -c break.lock .bzr.log*
.bzr.log:3
.bzr.log.old:45

Re: Locks on the Bzr repository

[Jason Rumney]

  On 20/8/2010 1:08 AM, Dan Nicolaescu wrote:
> My own locks.
> Sometimes commits fail (in a up to date tree!) after a long time, and
> then I have to remove my own locks to try again.

This happened to me too - I had to remove my own locks, despite the 
stack trace from the earlier failure suggesting that my locks had 
already been removed (by Stefan I presume).

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 19 Aug 2010 17:40:29 +0200
> 
> Could it be that bzr grabs the lock when you do "bzr commit" and keep it
> held for as long as you're editing the commit message?

Maybe so, but why would it take Jason 28 minutes and 50-odd seconds to
edit a commit message?

I suspect the culprit is network traffic, but Jason should be able to
tell more by looking at his .bzr.log file, where each operation is
logged together with its timing.

Re: Locks on the Bzr repository

[Jason Rumney]

  On 20/8/2010 12:24 AM, Eli Zaretskii wrote:
>> From: Stefan Monnier<monnier@iro.umontreal.ca>
>> Date: Thu, 19 Aug 2010 17:40:29 +0200
>>
>> Could it be that bzr grabs the lock when you do "bzr commit" and keep it
>> held for as long as you're editing the commit message?
> Maybe so, but why would it take Jason 28 minutes and 50-odd seconds to
> edit a commit message?
>
> I suspect the culprit is network traffic, but Jason should be able to
> tell more by looking at his .bzr.log file, where each operation is
> logged together with its timing.

Here are the commits I performed today:

The first one went quite smoothly but took about 8 minutes.

Thu 2010-08-19 22:50:16 +0800
0.058  bazaar version: 2.1.1
0.058  bzr arguments: [u'commit', u'-m', u"(w32_wnd_proc): Don't check 
context before initializing.\n", u'src/ChangeLog', u'src/w32fns.c']
0.086  looking for plugins in /home/jasonr/.bazaar/plugins
0.086  looking for plugins in 
/usr/lib/python2.6/dist-packages/bzrlib/plugins
0.159  encoding stdout as osutils.get_user_encoding() 'UTF-8'
0.225  opening working tree '/home/jasonr/bzr/emacs/trunk'
0.303  ssh implementation is OpenSSH
28.204  preparing to commit
[22809] 2010-08-19 22:51:09.255 INFO: Committing to: 
sftp://jasonr@bzr.savannah.gnu.org/srv/bzr/emacs/trunk/
52.688  Selecting files for commit with filter [u'src/ChangeLog', 
u'src/w32fns.c']
[22809] 2010-08-19 22:51:09.326 INFO: modified src/ChangeLog
[22809] 2010-08-19 22:51:09.344 INFO: modified src/w32fns.c
52.990  Using fetch logic to copy between 
CHKInventoryRepository('file:///home/jasonr/bzr/emacs/.bzr/repository/')(RepositoryFormat2a()) 
and 
CHKInventoryRepository('sftp://jasonr@bzr.savannah.gnu.org/srv/bzr/emacs/.bzr/repository/')(RepositoryFormat2a())
52.990  fetch up to rev {jasonr@gnu.org-20100819145109-17tqfx1uylo95ras}
[22809] 2010-08-19 22:58:21.431 INFO: Committed revision 101137.
487.501  return code 0


The second attempt failed, perhaps due to Stefan breaking the lock (as 
it failed in unlock self.confirm):

Thu 2010-08-19 23:07:28 +0800
0.052  bazaar version: 2.1.1
0.053  bzr arguments: [u'commit', u'-m', u'(set_frame_menubar): Remove 
call to undefined function.\n', u'src/ChangeLog', u'src/w32menu.c']
0.084  looking for plugins in /home/jasonr/.bazaar/plugins
0.085  looking for plugins in 
/usr/lib/python2.6/dist-packages/bzrlib/plugins
0.246  encoding stdout as osutils.get_user_encoding() 'UTF-8'
0.301  opening working tree '/home/jasonr/bzr/emacs/trunk'
0.606  ssh implementation is OpenSSH
29.123  preparing to commit
[23356] 2010-08-19 23:08:28.093 INFO: Committing to: 
sftp://jasonr@bzr.savannah.gnu.org/srv/bzr/emacs/trunk/
59.322  Selecting files for commit with filter [u'src/ChangeLog', 
u'src/w32menu.c']
[23356] 2010-08-19 23:08:28.202 INFO: modified src/ChangeLog
[23356] 2010-08-19 23:08:28.225 INFO: modified src/w32menu.c
59.631  Using fetch logic to copy between 
CHKInventoryRepository('file:///home/jasonr/bzr/emacs/.bzr/repository/')(RepositoryFormat2a()) 
and 
CHKInventoryRepository('sftp://jasonr@bzr.savannah.gnu.org/srv/bzr/emacs/.bzr/repository/')(RepositoryFormat2a())
59.631  fetch up to rev {jasonr@gnu.org-20100819150828-w2hw9ic7u7z9hfqp}
470.146  Raising exception with args ('Server connection dropped: ',)
470.146  Error suppressed by only_raises:
470.164  Traceback (most recent call last):
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 
222, in wrapped
     return unbound(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 319, 
in unlock
     self.confirm()
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 415, 
in confirm
     info = self.peek()
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 438, 
in peek
     info = self._read_info_file(self._held_info_path)
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 428, 
in _read_info_file
     return self._parse_info(self.transport.get_bytes(path))
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 435, in get_bytes
     f = self.get(relpath)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 431, in get
     failure_exc=errors.ReadError)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 721, in _translate_io_exception
     raise e
SSHException: Server connection dropped:

470.167  Error suppressed by only_raises:
470.208  Traceback (most recent call last):
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 
222, in wrapped
     return unbound(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 319, 
in unlock
     self.confirm()
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 415, 
in confirm
     info = self.peek()
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 438, 
in peek
     info = self._read_info_file(self._held_info_path)
   File "/usr/lib/python2.6/dist-packages/bzrlib/lockdir.py", line 428, 
in _read_info_file
     return self._parse_info(self.transport.get_bytes(path))
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 435, in get_bytes
     f = self.get(relpath)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 425, in get
     f = self._get_sftp().file(path, mode='rb')
   File "/usr/lib/pymodules/python2.6/paramiko/sftp_client.py", line 
245, in open
     t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
   File "/usr/lib/pymodules/python2.6/paramiko/sftp_client.py", line 
627, in _request
     num = self._async_request(type(None), t, *arg)
   File "/usr/lib/pymodules/python2.6/paramiko/sftp_client.py", line 
649, in _async_request
     self._send_packet(t, str(msg))
   File "/usr/lib/pymodules/python2.6/paramiko/sftp.py", line 172, in 
_send_packet
     self._write_all(out)
   File "/usr/lib/pymodules/python2.6/paramiko/sftp.py", line 136, in 
_write_all
     n = self.sock.send(out)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/ssh.py", line 
194, in send
     return self.__socket.send(data)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/ssh.py", line 
665, in send
     return os.write(self.proc.stdin.fileno(), data)
OSError: [Errno 32] Broken pipe

470.313  Traceback (most recent call last):
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 853, 
in exception_to_return_code
     return the_callable(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 
1055, in run_bzr
     ret = run(*run_argv)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 661, 
in run_argv_aliases
     return self.run_direct(**all_cmd_args)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 665, 
in run_direct
     return self._operation.run_simple(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 122, 
in run_simple
     self.cleanups, self.func, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 156, 
in _do_with_cleanups
     result = func(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 
3138, in run
     exclude=safe_relpath_files(tree, exclude))
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 
194, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree_4.py", line 
197, in commit
     result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 
194, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/mutabletree.py", line 
225, in commit
     *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commit.py", line 257, 
in commit
     possible_master_transports=possible_master_transports)
   File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 118, 
in run
     self.cleanups, self.func, self, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 156, 
in _do_with_cleanups
     result = func(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commit.py", line 420, 
in _commit
     self.branch.repository, new_revno, self.rev_id)
   File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 924, 
in import_last_revision_info
     self.set_last_revision_info(revno, revid)
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 
194, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 2508, 
in set_last_revision_info
     if self._get_append_revisions_only():
   File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 2691, 
in _get_append_revisions_only
     value = self.get_config().get_user_option('append_revisions_only')
   File "/usr/lib/python2.6/dist-packages/bzrlib/config.py", line 187, 
in get_user_option
     return self._get_user_option(option_name)
   File "/usr/lib/python2.6/dist-packages/bzrlib/config.py", line 733, 
in _get_user_option
     value = source()._get_user_option(option_name)
   File "/usr/lib/python2.6/dist-packages/bzrlib/config.py", line 406, 
in _get_user_option
     value = self._get_parser().get_value(section, option_name)
   File "/usr/lib/python2.6/dist-packages/bzrlib/config.py", line 997, 
in _get_parser
     return self._config._get_configobj()
   File "/usr/lib/python2.6/dist-packages/bzrlib/config.py", line 1505, 
in _get_configobj
     return ConfigObj(self._get_config_file(), encoding='utf-8')
   File "/usr/lib/python2.6/dist-packages/bzrlib/config.py", line 1500, 
in _get_config_file
     return StringIO(self._transport.get_bytes(self._filename))
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 435, in get_bytes
     f = self.get(relpath)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/sftp.py", 
line 425, in get
     f = self._get_sftp().file(path, mode='rb')
   File "/usr/lib/pymodules/python2.6/paramiko/sftp_client.py", line 
245, in open
     t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
   File "/usr/lib/pymodules/python2.6/paramiko/sftp_client.py", line 
627, in _request
     num = self._async_request(type(None), t, *arg)
   File "/usr/lib/pymodules/python2.6/paramiko/sftp_client.py", line 
649, in _async_request
     self._send_packet(t, str(msg))
   File "/usr/lib/pymodules/python2.6/paramiko/sftp.py", line 172, in 
_send_packet
     self._write_all(out)
   File "/usr/lib/pymodules/python2.6/paramiko/sftp.py", line 136, in 
_write_all
     n = self.sock.send(out)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/ssh.py", line 
194, in send
     return self.__socket.send(data)
   File "/usr/lib/python2.6/dist-packages/bzrlib/transport/ssh.py", line 
665, in send
     return os.write(self.proc.stdin.fileno(), data)
OSError: [Errno 32] Broken pipe

470.314  return code 3

By the time I'd tried again and failed due to my own stale locks (I'm 
not sure why - did bzr revive them after the above failure?), and then 
an out of date tree; updated the tree with Stefan's changes; and  
finally committed successfully, it had taken me 50 minutes to commit a 2 
line change!

Re: Locks on the Bzr repository

[Stefan Monnier]

> The second attempt failed, perhaps due to Stefan breaking the lock (as it
> failed in unlock self.confirm):

My breaking someone else's log happened a few days ago (can't remember
whose it was, sorry), so it's not the cause of this particular log.

> By the time I'd tried again and failed due to my own stale locks (I'm not
> sure why - did bzr revive them after the above failure?), and then an out of

So, IIUC the problem is that Bzr failed for some unknown reason and it
didn't bother to release the locks it had.  Sounds like a bug (indeed,
Bzr also fails to release locks when you interrupt it via C-c, which is
also a bug in my book).


        Stefan

Re: Locks on the Bzr repository

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 19 Aug 2010 23:42:54 +0200
> Cc: emacs-devel@gnu.org
> 
> So, IIUC the problem is that Bzr failed for some unknown reason and it
> didn't bother to release the locks it had.  Sounds like a bug (indeed,
> Bzr also fails to release locks when you interrupt it via C-c, which is
> also a bug in my book).

How about submitting a bug report to Bazaar developers?

Re: Locks on the Bzr repository

[Stefan Monnier]

>> So, IIUC the problem is that Bzr failed for some unknown reason and it
>> didn't bother to release the locks it had.  Sounds like a bug (indeed,
>> Bzr also fails to release locks when you interrupt it via C-c, which is
>> also a bug in my book).

> How about submitting a bug report to Bazaar developers?

It was already reported two years ago by someone else
https://bugs.launchpad.net/bzr/+bug/257217


        Stefan

Re: Locks on the Bzr repository

[Jay Belanger]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
...
> If that's indeed the culprit, I think it's a misfeature of Bzr, but at
> least we can workaround it by using "bzr commit -m <...>", which I would
> then *strongly* recommend.

I assume that "bzr commit -F ..." would work as well...

bzr send workflow (Was: Locks on the Bzr repository)

[Bojan Nikolic]

Hi All,

As already much discussed, the problems people are experiencing are
basically due to many people trying to write (in some way) to the same
sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/ location via the dumb sftp
protocol.

There are many ways around this, but one I did not see mentioned (maybe
I missed it) is using the bzr send command to send merge "bundles" via
email. This is the way bzr development itself worked while I followed
it, and I've used it on some of my own projects too. I was always very
impressed how well it worked.

Since maybe I missed the discussion on this, I just wanted to ask if
this workflow has been considered and rejected?  If not, I'd be happy to
offer a more detailed summary of how it works and the advantages.

I imagine one possible objection to this workflow has/will be the
unnecessary complication of merge history. One possible work around
would be an option to the merge bundle which basically indicated that it
should be rebased to the current trunk tip if this can safely be done.

If people are interested I would be happy to provide more details, at
least as far as I understand them.

Best,
Bojan

-- 
Bojan Nikolic          ||          http://www.bnikolic.co.uk

Re: bzr send workflow

[Stefan Monnier]

> Since maybe I missed the discussion on this, I just wanted to ask if
> this workflow has been considered and rejected?  If not, I'd be happy to
> offer a more detailed summary of how it works and the advantages.

Before we can switch to such a system, we need to get rid of the
ChangeLog files (which are otherwise a constant source of spurious
conflicts which make such a workflow impractical).  I haven't dared to
enter this discussion yet.


        Stefan

Re: bzr send workflow

[Bojan Nikolic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Before we can switch to such a system, we need to get rid of the
> ChangeLog files (which are otherwise a constant source of spurious
> conflicts which make such a workflow impractical).  I haven't dared to
> enter this discussion yet.

I see. I'm sure this will not satisfy everybody, but the --gnu-changelog
option to bzr log attempts to produce something fairly close to the
ChangeLog files, at least in format. For example:

bzr log -n 1 -r -10.. --gnu-changelog  lisp/

gives a reasonable summary of last 10 changes in lisp directory.

Best,
Bojan


-- 
Bojan Nikolic          ||          http://www.bnikolic.co.uk

Re: No ChangeLog's (was: bzr send workflow)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sun, 22 Aug 2010 00:05:33 +0200
> Cc: emacs-devel@gnu.org
> 
> > Since maybe I missed the discussion on this, I just wanted to ask if
> > this workflow has been considered and rejected?  If not, I'd be happy to
> > offer a more detailed summary of how it works and the advantages.
> 
> Before we can switch to such a system, we need to get rid of the
> ChangeLog files (which are otherwise a constant source of spurious
> conflicts which make such a workflow impractical).  I haven't dared to
> enter this discussion yet.

Do we have the necessary infrastructure for ditching ChangeLog files?

Working on a complex feature could take quite some time, perhaps more
than a single Emacs session.  With ChangeLog's, one can keep the log
of changes in a disk file, which is persistent, and then reuse that
log at commit time.  What do we do instead when there are no ChangeLog
files?  Are there any commands that let us edit log messages for
future commits, keep their record, and use them conveniently when the
time comes to commit?

Re: No ChangeLog's (was: bzr send workflow)

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > What do we do instead when there are no ChangeLog files?

You keep a temporary ChangeLog file, just not under version control.

More complex workflows could use the "pipeline" or "loom" plugins,
which I think have features for keeping track of log messages while
development proceeds.

 > Are there any commands that let us edit log messages for future
 > commits, keep their record, and use them conveniently when the time
 > comes to commit?

This sounds like job for Emacs to me.  The basic workflow for a commit
is

1.  move relevant log entry to .msg
2.  bzr commit --file=.msg    # assumes commitables already bzr add'ed
3.  rm .msg

I suspect that a simple command that

1.  looks for a ChangeLog
    if not found, edits .msg directly, then does 2 and 3 of the basic
    workflow
2.  marks the top log entry
3.  asks for confirmation y/n
    if not confirmed, drop into a recursive edit allowing user to mark
    a different log entry or perhaps edit it, then confirm with C-c C-c
4.  do 1, 2, and 3 of the basic workflow above

might be a good UI that could even be added to vc.el.

Re: No ChangeLog's (was: bzr send workflow)

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>     bojan@bnikolic.co.uk,
>     emacs-devel@gnu.org
> Date: Sun, 22 Aug 2010 17:11:15 +0900
> 
>  > Are there any commands that let us edit log messages for future
>  > commits, keep their record, and use them conveniently when the time
>  > comes to commit?
> 
> This sounds like job for Emacs to me.

Yes, of course.  I was just asking if we already have that somewhere.
log-edit.el doesn't, unless I missed something.

Re: bzr send workflow

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Before we can switch to such a system, we need to get rid of the
> ChangeLog files (which are otherwise a constant source of spurious
> conflicts which make such a workflow impractical).

Pretty much a non-issue with git-merge-changelog (from gnulib).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: bzr send workflow

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Sun, 22 Aug 2010 14:29:00 +0200
> Cc: Bojan Nikolic <bojan@bnikolic.co.uk>, emacs-devel@gnu.org
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > Before we can switch to such a system, we need to get rid of the
> > ChangeLog files (which are otherwise a constant source of spurious
> > conflicts which make such a workflow impractical).
> 
> Pretty much a non-issue with git-merge-changelog (from gnulib).

You mean, use it with bzr via the extmerge plug-in?

And btw, it sounds a simple job to write an Emacs command that would
resolve such conflicts in a ChangeLog file and commit the result.

Re: bzr send workflow

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Date: Sun, 22 Aug 2010 14:29:00 +0200
>> Cc: Bojan Nikolic <bojan@bnikolic.co.uk>, emacs-devel@gnu.org
>> 
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> 
>> > Before we can switch to such a system, we need to get rid of the
>> > ChangeLog files (which are otherwise a constant source of spurious
>> > conflicts which make such a workflow impractical).
>> 
>> Pretty much a non-issue with git-merge-changelog (from gnulib).
>
> You mean, use it with bzr via the extmerge plug-in?

I've only used it with git so far.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: bzr send workflow

[Richard Stallman]

There is a new GNU package called vc-changelog.
Does it address the issues for Emacs concerning ChangeLog files
and bzr?

Re: bzr send workflow

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 23 Aug 2010 00:24:56 -0400
> Cc: bojan@bnikolic.co.uk, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Reply-To: rms@gnu.org
> 
> There is a new GNU package called vc-changelog.
> Does it address the issues for Emacs concerning ChangeLog files
> and bzr?

Where do I find this package?  It's not in the GNU Software
Directory.  The only one that comes close is vc-dwim -- did you mean
this one?

Re: bzr send workflow

[Eli Zaretskii]

> From: Eli Zaretskii <eliz@gnu.org>
> Date: Mon, 23 Aug 2010 01:11:04 -0400
> Cc: emacs-devel@gnu.org
> Reply-To: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Richard Stallman <rms@gnu.org>
> > Date: Mon, 23 Aug 2010 00:24:56 -0400
> > Cc: bojan@bnikolic.co.uk, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> > Reply-To: rms@gnu.org
> > 
> > There is a new GNU package called vc-changelog.
> > Does it address the issues for Emacs concerning ChangeLog files
> > and bzr?
> 
> Where do I find this package?  It's not in the GNU Software
> Directory.  The only one that comes close is vc-dwim -- did you mean
> this one?

I guess you meant vc-chlog that is part of vc-dwim.

FWIW, I don't think it addresses the problems raised in this thread
regarding the ChangeLog files.  Those problems are due to bzr (and
most if not all other VCSs) not being smart enough when merging
ChangeLog files.  This results in conflicts when merging changes,
whose resolution is a mechanical job of removing the conflict markers
and moving the log entries to the top of the file.

It shouldn't be hard to write an Emacs command to resolve these
conflicts.  But it would be best if bzr itself could merge ChangeLog
files without producing the conflicts in the first place.

vc-chlog does not try to deal with that.  Its main goal seems to be to
make it easier to prepare ChangeLog entries before you are going to
commit a changeset.

Re: bzr send workflow

[Richard Stallman]

    Where do I find this package?  It's not in the GNU Software
    Directory.

Perhaps it has not had time to be entered.  It became a GNU package
last week.

Simon Josefsson <simon@josefsson.org> can tell you more about it.

Re: bzr send workflow

[Andy Wingo]

Greets,

On Sat 21 Aug 2010 18:05, Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Before we can switch to such a system, we need to get rid of the
> ChangeLog files (which are otherwise a constant source of spurious
> conflicts which make such a workflow impractical).  I haven't dared to
> enter this discussion yet.

That I am aware of, you have two good options:

 * Keep writing changelogs, pasting them into the commit logs as you do
   already, and add special support to merging changelogs (e.g.:
   git-merge-changelog from gnulib, perhaps ported to bzr if you stay
   with bzr)

 * Stop writing changelogs, and generate changelogs via a
   dist-hook. Guile does this now, using gitlog-to-changelog from
   gnulib.

Cheers,

Andy
-- 
http://wingolog.org/

Re: bzr send workflow

[Juanma Barranquero]

On Wed, Aug 25, 2010 at 17:03, Andy Wingo <wingo@pobox.com> wrote:

>  * Stop writing changelogs, and generate changelogs via a
>   dist-hook. Guile does this now, using gitlog-to-changelog from
>   gnulib.

At this moment, the quality of our ChangeLog entries is quite superior
to the quality of commit log messages, so that'd be a net loss.

I'm no big fan of keeping ChangeLogs *and* detailed commit logs, but
I'm even less fan of non-modifiable commit logs.

    Juanma

Re: bzr send workflow

[Chong Yidong]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Wed, Aug 25, 2010 at 17:03, Andy Wingo <wingo@pobox.com> wrote:
>
>>  * Stop writing changelogs, and generate changelogs via a
>>   dist-hook. Guile does this now, using gitlog-to-changelog from
>>   gnulib.
>
> At this moment, the quality of our ChangeLog entries is quite superior
> to the quality of commit log messages, so that'd be a net loss.
>
> I'm no big fan of keeping ChangeLogs *and* detailed commit logs, but
> I'm even less fan of non-modifiable commit logs.

Anyway, this should be a non-issue for Emacs users.  Just write a
ChangeLog in the usual way; then, before committing, do C-c C-a to pull
it into the *VC-log* buffer and edit as necessary.

Re: bzr send workflow

[Juanma Barranquero]

On Thu, Aug 26, 2010 at 19:11, Chong Yidong <cyd@stupidchicken.com> wrote:

> Anyway, this should be a non-issue for Emacs users.

You're right. Should be.

    Juanma

Re: bzr send workflow

[Stephen J. Turnbull]

Chong Yidong writes:

 > Anyway, this should be a non-issue for Emacs users.  Just write a
 > ChangeLog in the usual way; then, before committing, do C-c C-a to
 > pull it into the *VC-log* buffer and edit as necessary.

You're missing the point.  Once you've done that, and committed, the
log of that revision cannot be changed.  That means that users of bzr
must do some fancy footwork to ensure that the logs that are pushed
are appropriate, or they must wait until the patch has been approved
*including* ChangeLog to commit and push, or they need to pollute the
history with revisions that simply fix up the ChangeLog.

For people who use branches even for trivial one-commit work, the
footwork needed is not at all fancy.  For people who commit in a bound
branch, it's highly discouraged, for the same reason rebasing public
branches is discouraged -- the only option is a new commit that does
nothing except fix the log.

Re: bzr send workflow

[Wojciech Meyer]

[-- Attachment #1: Type: text/plain, Size: 787 bytes --]

Hi,

Generally I would suggest two things for Change-logs:
- if anybody plan to generate it from bzr log, then maybe keeping the old
one is OK and prepending a new one. There would not be anything lost in this
case (as suggested - quality of commit comments vs changelog)
- now, we could tag textually commit messages, saying this is `commit'
significant and should be put into Change-Log (or just keeping the revision
numbers lists in separate place, this would be less appealing, change of vc,
maintaining another file).

The only drawback of using tagged commits, is that they might not reflect
what really happened at *this* particular commit, otherwise it solves the
problem (at least with ChangeLogs).

Mine two cents (sorry about getting into the middle of discussion).

Wojciech

[-- Attachment #2: Type: text/html, Size: 840 bytes --]

bzr send workflow (Was: Locks on the Bzr repository)

[Stephen J. Turnbull]

Bojan Nikolic writes:

 > As already much discussed, the problems people are experiencing are
 > basically due to many people trying to write (in some way) to the
 > same sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/ location via the
 > dumb sftp protocol.
 > 
 > There are many ways around this, but one I did not see mentioned
 > (maybe I missed it) is using the bzr send command to send merge
 > "bundles" via email. This is the way bzr development itself worked
 > while I followed it, and I've used it on some of my own projects
 > too. I was always very impressed how well it worked.

I don't think this helps much.  The underlying problem for Emacs (and
other bzr-based projects on Savannah) is that Savannah doesn't want to
allow bzr+ssh until the Savannah admins understand the security
implications.  The Bazaar development workflow involves at least one
separate server application not yet available on Savannah (the pqm
Patch Queue Manager), and I can't imagine that installing that would
be any easier than upgrading already installed software to use a smart
server over ssh.

Re: bzr send workflow (Was: Locks on the Bzr repository)

[Lennart Borgman]

On Sun, Aug 22, 2010 at 9:56 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Bojan Nikolic writes:
>
>  > As already much discussed, the problems people are experiencing are
>  > basically due to many people trying to write (in some way) to the
>  > same sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/ location via the
>  > dumb sftp protocol.
>  >
>  > There are many ways around this, but one I did not see mentioned
>  > (maybe I missed it) is using the bzr send command to send merge
>  > "bundles" via email. This is the way bzr development itself worked
>  > while I followed it, and I've used it on some of my own projects
>  > too. I was always very impressed how well it worked.
>
> I don't think this helps much.  The underlying problem for Emacs (and
> other bzr-based projects on Savannah) is that Savannah doesn't want to
> allow bzr+ssh until the Savannah admins understand the security
> implications.  The Bazaar development workflow involves at least one
> separate server application not yet available on Savannah (the pqm
> Patch Queue Manager), and I can't imagine that installing that would
> be any easier than upgrading already installed software to use a smart
> server over ssh.


Is moving to Launchpad an option?

Re: bzr send workflow

[Miles Bader]

Lennart Borgman <lennart.borgman@gmail.com> writes:
>> I don't think this helps much.  The underlying problem for Emacs (and
>> other bzr-based projects on Savannah) is that Savannah doesn't want to
>> allow bzr+ssh until the Savannah admins understand the security
>> implications.
>
> Is moving to Launchpad an option?

If we're going to stop using "something GNU" because of technical
issues, it'd be far better to stop using bzr and stick with savannah,
than stick with bzr and stop using savannah.  The problems with bzr go
far deeper than the protocol used to talk to the server.

-miles

-- 
A zen-buddhist walked into a pizza shop and
said, "Make me one with everything."

Re: bzr send workflow

[Bojan Nikolic]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Bojan Nikolic writes:
>
>  > As already much discussed, the problems people are experiencing are
>  > basically due to many people trying to write (in some way) to the
>  > same sftp://bzr.sv.gnu.org/srv/bzr/emacs/trunk/ location via the
>  > dumb sftp protocol.
>  > 
>  > There are many ways around this, but one I did not see mentioned
>  > (maybe I missed it) is using the bzr send command to send merge
>  > "bundles" via email. This is the way bzr development itself worked
>  > while I followed it, and I've used it on some of my own projects
>  > too. I was always very impressed how well it worked.
>
> I don't think this helps much.  The underlying problem for Emacs (and
> other bzr-based projects on Savannah) is that Savannah doesn't want to
> allow bzr+ssh until the Savannah admins understand the security
> implications.  The Bazaar development workflow involves at least one
> separate server application not yet available on Savannah (the pqm
> Patch Queue Manager), and I can't imagine that installing that would
> be any easier than upgrading already installed software to use a smart
> server over ssh.

My idea was that in the first instance the bundles would be merged by (a
select group of) people. These people would either have a fast
connection to the bzr.sv.gnu.org, or even better, a login to the machine
hosting this branch, in which case the merges could be done without any
network traffic. It is a bit of an overhead but it takes only a few
seconds to do a merge of bundle from Gnus.  This has the additional
benefit that most changes would have four eyes on them before being
committed to the trunk.

To reiterate the main benefit though: accepting "bzr send" bundles means
that people on even the slowest connections can contribute easily and
efficiently. I've used these satisfactorily over slow GPRS connections.

Best,
Bojan


-- 
Bojan Nikolic          ||          http://www.bnikolic.co.uk

Re: bzr send workflow

[Stephen J. Turnbull]

Bojan Nikolic writes:

 > My idea was that in the first instance the bundles would be merged
 > by (a select group of) people.

Sure, and who do you mean by "people"?  Presumably Stefan, Yidong, et
al.  I don't think that having Stefan merge Eli's changes is efficient
use of anybody's time.  Use of bundles might make contribution by
occasional contributors more efficient, but I don't think it addresses
the problem that some core developers are bottlenecking on bzr/Savannah.

 > To reiterate the main benefit though: accepting "bzr send" bundles
 > means that people on even the slowest connections can contribute
 > easily and efficiently.  I've used these satisfactorily over slow
 > GPRS connections.

AFAIK Stefan et al are already as willing to accept merge directives
as any other form of patch.  It wouldn't hurt to advertise that
option, of course.  But I really don't think it's a solution to the
current bottleneck.

Re: bzr send workflow

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Mon, 23 Aug 2010 13:14:25 +0900
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> I don't think that having Stefan merge Eli's changes is efficient
> use of anybody's time.

(and vice versa)  I agree completely.

> AFAIK Stefan et al are already as willing to accept merge directives
> as any other form of patch.  It wouldn't hurt to advertise that
> option, of course.  But I really don't think it's a solution to the
> current bottleneck.

100% agreement.

Unless we have a clerk whose job is committing bundles sent by others,
and unless that clerk is on a very fast link, this suggestion will not
resolve the root cause of the problems being discussed here.