Re: Locks on the Bzr repository

["Stephen J. Turnbull" <stephen@xemacs.org>]

Eli Zaretskii writes:

 > Well, Stephen refused to tell more, citing my imaginary
 > unwillingness to know.  (Since when asking a question means you
 > don't want to know the answer?)

When you consistently ask for details before you have enough
understanding of the theory, or at least the pragmatic constraints.

Anyway, I don't use bzr in anger so my actual workflows aren't going
to be particularly useful.

 > So I guess we will never know what Stephen was talking about.

No, I changed my mind about posting to the thread since Jan mentioned
BzrForEmacsDevs and the recommendation for binding the mirror branch.

 > Btw, pull is only possible when the branch didn't diverge from
 > upstream, which means it is not suited well for a branch where you
 > do development.

D'oh.  You just don't ever want to do development in your mirror
branch (unless it's bound).  The reason this is a problem in bzr (and
to some extent hg) is that branches are heavyweight; you don't want
scads of them being made automatically for internal operations.  So
the user has to make mirror branches explicitly when needed.  git
makes branching extremely lightweight, and therefore need not hesitate
to use mirror ("tracking") branches which are read-only to the user,
and hardly visible.

 > Merging into a local branch is what I had in mind,

Really, I'm coming to believe that bzr is fundamentally unsuited to
radically distributed development (which Emacs approximates, since it
has a fairly large group of committers and no required synchronization
protocol).  Merging into a local branch should not require thought!

 > As for rebase, I don't really understand what you suggest, and how
 > it will help with an unbound branch where you develop.  Perhaps you
 > could elaborate.  (I know what's a rebase and how to do it with
 > Bazaar.)

The workflow should be something like

0.  work in "work", a branch of "mirror" (local) which is in turn a
    branch of "trunk" (on Savannah)
1.  pull from "trunk" into "mirror"
2.  rebase "work" on "mirror"
3.  repeat 0-2 until done
4.  cd to mirror & pull from "work" into "mirror" (this will always succeed!)
5.  push from "mirror" to "trunk"
6.  if 5 succeeds, you're done (for now, go to 0 to start the next change)
7.  else pull --overwrite from "trunk" into "mirror"
8.  goto 2

 > Sure.  But that's in theory.  In practice, you also need to
 > consider what kind of changes you make locally.  At least for me,
 > there are 2 main kinds: bug-fixes and development of significant
 > new features.  (There's a third kind as well: when I need to make
 > changes for the MS-DOS build, but I don't think this part will be
 > interesting for anyone but myself.)

That's interesting in itself for anybody maintaining an optional
feature or different platform, I should think.

The other thing is that you have multiple development branches.  This
is where the rebase-based workflow is attractive to me.  The reason is
that I generally want to have an easy way to diff against the branch
parent (so I don't want to merge into the local branch and have mixed
sequences of trunk and local commits, even if the VCS could
theoretically handle it) but I do want my branch to incorporate the
latest trunk commits.

I guess this could also work with the mirror being a branch bound to
"trunk", but it makes me nervous because of the incompleteness of bzr
documentation; often it's not obvious what checkouts (lightweight or
heavyweight) will do in any given situation.  Also, you could have a
script that will do the rebase step in several related branches with
one pull, avoiding unnecessary network traffic.

 > For the first kind, it is IMO not very good to have the changes
 > uncommitted upstream for prolonged periods of time,

What's your definition of "prolonged"?  I'm thinking in terms of
simply committing all changes locally until the end of the work
session, then doing the merge to mirror and push to upstream in a
batch at the end of the session.

 > because people are using the development code, and the bugs annoy
 > them.  So you'd like to commit those to upstream frequently.  And
 > if you commit frequently, the proposed rebase-push workflow (IIUC)
 > is not a real improvement, perhaps even a nuisance, because it adds
 > the overhead of the rebase and does not save me from frequent slow
 > commits to the master repository.

How frequently do you commit?  How finely do you divide changesets?  I
will often make 5 to 10 commits in a session when I'm working on minor
bugs.  I really would not like it if I had to wait even one minute for
a commit of a typo fix.  (I always separate typo fixes from "real bug"
fixes, and although I do batch typo fixes when I make them one after
another, I don't rebase just so that I can group all typo fixes in a
single commit.)  At least when I push to my launchpad branch of
Mailman, there are many seconds of setup cost, which is only incurred
once per push.  And then I go for coffee or dinner or read mail.  (Of
course, I don't have a race condition since it's a private branch.
But I do have to check for success.)

 > And if your local commits have bug-fixes and new features
 > interspersed, and you want to commit just the bug-fixes, won't the
 > workflow you need for such cherry-picking become even more
 > complicated?

Yes.  I don't do that in bzr.  (It's no big deal in git because of the
interactive rebase feature.)

 > > With bound branches, your branch is locked up until the commit
 > > goes through.  You can't do anything while you have uncommitted
 > > changes in your source.

 > Why would you say that?  That's not true.  Nothing prevents me from
 > editing while "bzr ci" churns away.  The system is not locked, only
 > (some) bzr operations will fail.  But most of the development is
 > not about bzr ops, its about using the editor, the compiler, and
 > other development tools, none of which are locked up when "bzr ci"
 > runs.

No, but if you do a typo fix, commit, then find another typo, 50
minutes (according to the recent report) is a very long time to
wait....  I imagine it would also be a bad idea to continue to work on
any of the files currently being committed, as if the commit fails you
need to disentangle the changes by hand.  bzr is no help because it has
no record of which changes are part of the intended commit, and which
ones aren't.  You could avoid saving your post-bzr-ci work, and use
the difference between the buffer and the file, but that seems a bit
fragile.

 > I don't understand this argument at all.  In fact, I think it's
 > plain false.  My workflow in a bound branch is this:
 > 
 >   brz up
 >   [build the current upstream]
 >   [some minimal sanity checks to make sure upsteam works]
 >   [make changes]
 >   [build and test the modified binary]
 >   [repeat the sanity checks, fix anything that became broken]
 >   [bzr up]
 >   [if there are any changes upstream, build and test again]
 >   [repeat last two steps until "bzr up" brings no changes]
 >   bzr ci
 > Now please tell me how can I commit code that is "untested and
 > unsafe" with this workflow?  What am I missing?

That workflow is safe, because the last thing you do before committing
is test (the very last bzr up is a no-op, of course).  That is what is
required for a safe workflow.  But it's very tempting to not do tests
after bzr up succeeds without conflicts, and experience in many
projects is that it is a common practice to omit testing after a merge.

 > > It also occurs to me that, by asking everybody to use bound
 > > branches, you have massively increased the contention on your
 > > public repo and the server.  That is making your problems worse.

That is definitely not true.  The recommended workflow (at least as
Karl and I wrote it) assumed that "one-commit changes" would be
performed in a separate branch, and merged into the mirror (bound
branch) in batches.  Thus it's equivalent to a pull/rebase/push
workflow for the purpose of hits on the public repo.

It was others who insisted on recommending work *in* the bound branch,
which could increase the burden on the server (but I don't know how
much).  IIRC, however, that is documented in a separate wiki page, and
not as part of the BzrForEmacsDevs recommended workflow.

However, I doubt that's really a big problem.  The real problem is
abysmal administration at Savannah (at least for this purpose), and
maybe bzr's inherent slowness.  I don't understand the stale locks
problem; at least it's not frequently reported on the bazaar list.
(That may be because so few people use dumb servers for the trunk
repository, if it's a bug restricted to dumb servers.)