Turning in quality work, the goal of merging

Turning in quality work, the goal of merging

[Mike Mattie]

[-- Attachment #1.1: Type: text/plain, Size: 8239 bytes --]

Hello,

-> Intro

I would like to add some relevant ideas to the discussion of merging which has devolved into tools.
I tried to shy away from the discussion but it's hard when you are being discussed in the third
person. I am the third person in that I am very likely younger than most of the people on this
list. 

Having said that age is irrelevant to the issue, the only real difference being the music
I enjoy. Everything else is the same, an unpopular foreign war and lots of anti-war lyrics.
Oh yeah, bottled water, that's new :)

-> Meat

Much of the debate so far has devolved into trying to explain one tool in terms of another,
without much discussion of what the goals, or criteria for merging is.

What would be useful IMHO is to articulate what it is to turn in quality work for review
and inclusion in a code-base. With a high standard for quality clearly fixed there is
opportunity for Emacs to innovate beyond the current tools if necessary and take the
lead in excellence for no other reason than our own desire to benefit from that excellence.

So here is a sketch of what a programmer who happens to be a bit younger thinks of merging.

Quality in merging is preparing the work you have done for review and integration. Not a single
tool out there supports the developer before he publishes his changes.

My ideal scenario is to first conceive an idea. I will hack away at it from several different
angles until a development plan forms in my mind. A development plan and a merge plan are
the same thing: a sequence of steps that incrementally construct the desired changes.

A good interface for supporting this would have something like the speedbar on the side
of a development window. In this side-window I would create and maintain my plan. Each
of the steps in the plan would be a change-log entry in progress. It would describe
succinctly and completely the purpose of each change-set, significant design and implementation
properties such as algorithms I have chosen, vital knowledge discovered while exploring the
problem domain, and impact on the existing code-base.

While I am developing the code essentially ad-hoc (as is all development unless you have solved
that problem before) I am committing changes. The changelog for these changes is essentially
irrelevant to the merge and largely useless because it is likely a long sequence of "fixed this",
"spelling fixes", "added function foo".

What I need in a tool is a facility to clean up my ad-hoc change history into the merge plan
that is also serving as a tool to keep my eye on the ball. Say I have decided on a three step
plan to add a feature. I notice that other parts of the code-base have to be changed as well,
or will benefit from using some of my code.

My first step is to introduce as much of my code as possible as dead-code and make sure nothing
is affected AFAICT. The second step is to make any modifications necessary to the existing
code-base that aren't intrinsically linked to new features, and verify that I haven't
introduced regressions. My third step is to add my new feature.

I think any merge should have this as the basic plan as it mitigates risk to the existing code
at each step, and takes care to ensure that it places the least burden possible on the reviewers.
Each of these steps needs to be testable in my working-copy to check that each step compiles,
does not have regressions, and missing pieces.

When it looks like my new code is ready to merge I run into a big problem that existing tools
don't even address much less resolve. My goal is to merge in three steps: M1, M2, M3.
If I have a function "foo" that needs to go in step 1 or M1 I have a problem. The changes
to function foo are scattered throughout the chronology of my revision history. What I need
to do is re-order my revision history to organize it properly.

A interface for doing such a thing would ideally have my plan in one window, and my change
history in another. I would like to be able to sort the changes into the merge plan by selecting
a change and "attaching" it to one of my merges.

M1 (changelog text)
 |-> change
 |-> change

M2
 |-> change
 |-> change

If I attach my introduction of function foo into step M1, it is also going to show up in
M2, and M3. Now later in the chronology of my revisions I have various fixes for function foo.
I want to attach those changes to M1, and have it automatically adjust the diffs for M2, M3,

The only tool that is even close to something like this is darcs with it's theory of commutation
of patches. Now when you consider the possibility of code moving around it gets really hairy
and you might assume it is impossible. It is not, at least with Emacs. It is theoretically
possible that Emacs can track the movement of code, where tracking is being able to for
a given region of code, identify it's location in any given revision starting from where it
was introduced. In fact Emacs could even track copy and paste in theory.

Assuming that my hypothetical tool has allowed me to re-organize my change history into
my merge plan I can now commit a coherent planned merge sequence. Even better I was able
to incrementally develop my merge plan and merge change-logs while I was developing instead
of trying to write it down at one sitting, largely from memory and tedious review.

Now that I have finished the private development phase I need to get review. At this
point I need to commit on a branch, as any multiple change commit does to remain atomic.

This introduces two new kinds of changes that I need to organize into my prepared merge.
First is various fixes and changes I will get from review. Assuming I have my tool
for re-organizing my revisions this is no problem. To actually merge though I need
to re-base against the latest revision.

Rebasing introduces M0, changes that reconcile my merge sequence of M1,M2,M3 against
the head. This stage is not something I want to expose for review, it is the tool's
internal representation of rebase. It is special in that it doesn't have the benefit
of being able to track the movement of code as Emacs can in my working repository.

Assuming I have successfully passed review and rebased, I need to commit. At this
point I need to deal with the project's version control system. A key point is that
which VCS system in use is *entirely* random. I may work on a dozen different projects
and each one has chosen their own VCS for their own reasons.

If the hypothetical tool I have outlined above is built into the VCS it's nearly
useless since integrating my changes into their code should not be predicated
on the necessity of convincing them to switch their entire project over to
the mythical *one true VCS to rule them all*. Even assuming that the current
debate results in switching to a new VCS that doesn't help any of the Emacs
developers on all the other projects they are involved in.

Questions:

1. Do the Emacs developers consider the above description to be a good standard or goal
   for merging practices.

2. What is the existing standard for merging ? Is it compatible, better, or worse than
   what I have described. (I am all ears for the lessons learned by a project at rev 23)

3. Can existing tools be used to implement this work-flow ?

4. Can new tools be built to facilitate this work-flow with any given VCS system ?
   ie can vc be hacked towards this direction if the Emacs developers so desire ?

In closing I will use any tool that let's me reach a higher standard of quality
than I could without it, and I never get to choose the tools unless it's my
project, or my machine. If something cool does emerge from this debate I hope
it runs in Emacs because that's the only thing I can count on. Everything else
was determined by historical accident and fiat long before I even laid eyes
on the project.

One request: If this does spark ideas for new vc features please start another
topic. It would be very nice if this thread continues, to remain a discussion
of defining quality merging. Tools help but are not the mechanism, not the goal.

humbly,
Mike Mattie

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Turning in quality work, the goal of merging

[Thien-Thi Nguyen]

() Mike Mattie <codermattie@gmail.com>
() Sun, 6 Jan 2008 06:54:17 -0800

   Tools help but are not the mechanism, not the goal.

tools are not a lot of things.

thi