The git mirror is *very* badly screwed up

The git mirror is *very* badly screwed up

[Eric S. Raymond]

While investigating the git mirror with reposurgeon. I spent most of
the last two days chasing some odd behavior in reposurgeon that I
thought was a bug.  But it turns out not to be a bug at all; rather,
there's a kind of malformation in the mirror I've never seen
before.

Look at this span of commits:

reposurgeon% 867? inspect
Event 865 ===============================================================
commit refs/tags/v0.3.0
mark :860
author Chris Wanstrath <chris@ozmm.org> 1268028778 -0800
committer Chris Wanstrath <chris@ozmm.org> 1268028778 -0800
data 13
Just in case
from :858
M 100644 :859 coffee-mode.el

Event 867 ===============================================================
commit refs/tags/v0.3.0
mark :862
author Chris Wanstrath <chris@ozmm.org> 1268028809 -0800
committer Chris Wanstrath <chris@ozmm.org> 1268028809 -0800
data 6
0.3.0
from :860
M 100644 :861 coffee-mode.el

Event 869 ===============================================================
commit refs/tags/sml-mode-6.0
mark :864
author Sergei Lebedev <superbobry@gmail.com> 1269545595 -0700
committer Chris Wanstrath <chris@ozmm.org> 1269545595 -0700
data 60
Fix hanging of `coffee-previous-indent` at start of buffer.
from :862
M 100644 :863 coffee-mode.el

Event 874 ===============================================================
commit refs/tags/sml-mode-6.0
mark :869
author tav <tav@espians.com> 1269545212 +0800
committer Chris Wanstrath <chris@ozmm.org> 1269545212 +0800
data 64
Change so that the compiled buffer doesn't open in a new frame.
from :862
M 100644 :868 coffee-mode.el

This looks pretty normal - until you notice that events 869 and 874,
which should be sequential commits, are both parented by event 867.
This is badly wrong; either 874 should be parented on 869 (almost
certainly correct) or 869 and 874 should have different branch fields.

I had wondered why the parent links in the early part of a gitk
listing of the Emacs history looked so spaghettilike.  Now I think
many of them are wrong, and this also explains the persistent problems
reposurgeon has been exhibiting with junk lightweight tags that can't
be deleted.  (The bad links defeat reposurgeon's algorithm for
reassigning branch fields.)

I'm investigating possible remedies.  Because this kind of
malformation is easy to spot once you know what to look for, 
it may be possible for me to write an automated fixer in 
reposurgeon.

Andreas, how exactly are you doing the mirroring from Bazaar?  Is
there any chance parent links are being scrambled at that point?

The other possibility is that parsecvs botched the CVS lift really badly.
That wouldn't be good news for me either, because if so cvs-fast-export
probably inherited the bug. 
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

He that would make his own liberty secure must guard even his enemy from
oppression: for if he violates this duty, he establishes a precedent that
will reach unto himself.	-- Thomas Paine

Re: The git mirror is *very* badly screwed up

[Andreas Schwab]

esr@thyrsus.com (Eric S. Raymond) writes:

> This looks pretty normal - until you notice that events 869 and 874,
> which should be sequential commits, are both parented by event 867.
> This is badly wrong; either 874 should be parented on 869 (almost
> certainly correct) or 869 and 874 should have different branch fields.

These numbers are useless.  Use git revisions.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: The git mirror is *very* badly screwed up

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> esr@thyrsus.com (Eric S. Raymond) writes:
> 
> > This looks pretty normal - until you notice that events 869 and 874,
> > which should be sequential commits, are both parented by event 867.
> > This is badly wrong; either 874 should be parented on 869 (almost
> > certainly correct) or 869 and 874 should have different branch fields.
> 
> These numbers are useless.  Use git revisions.

Those aren't bzr revisions, they're marks from a git stream dump.  If they're
messed up, then the parent links in the live git repo are messed up too.

I ran some auditing code on the first 1100 or so commits.  This is what
came out:

wrong parent links under :417
wrong parent links under :503
wrong parent links under :512
wrong parent links under :520
wrong parent links under :528
wrong parent links under :613
wrong parent links under :620
wrong parent links under :624
wrong parent links under :626
wrong parent links under :629
wrong parent links under :637
wrong parent links under :641
wrong parent links under :646
wrong parent links under :651
wrong parent links under :655
wrong parent links under :862
wrong parent links under :894
wrong parent links under :926
wrong parent links under :934
wrong parent links under :939
wrong parent links under :947
wrong parent links under :966
wrong parent links under :975
wrong parent links under :1011
multiple root commits: set(['commit@:676', 'commit@:1017', 'commit@:75', 'commit@:579', 'commit@:39', 'commit@:2'])

Bottom line is, the topology is badly screwed up.  I already knew about the
multiple root commits; I'm fairly sure they relate to these junk branches

    21	commit	refs/tags/lost+found/d4cd8d7c34cc79dc4abd0fec33cec74afa33e2d0
  1723	commit	refs/tags/lost+found/12d398afa4d2b77d606b28fe43a2e5200c863419
  3595	commit	refs/tags/lost+found/e387e14faeea08877c99ddf798d5a4c9f7999c72
  3763	commit	refs/tags/lost+found/9a1501510c2a842e8597d695fec05563161a84b7
  3773	commit	refs/tags/lost+found/5cf718038f760d6dc2e21dee3ce98061a975488a

which I was going to delete anyway.  They seem to consist entirely of dummy
test commits by Glen Morris.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: The git mirror is *very* badly screwed up

[Andreas Schwab]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> Those aren't bzr revisions, they're marks from a git stream dump.

That's why they are useless.  Tell me the git revisions so I can tell
you what went wrong.

> Bottom line is, the topology is badly screwed up.  I already knew about the
> multiple root commits; I'm fairly sure they relate to these junk branches
>
>     21	commit	refs/tags/lost+found/d4cd8d7c34cc79dc4abd0fec33cec74afa33e2d0
>   1723	commit	refs/tags/lost+found/12d398afa4d2b77d606b28fe43a2e5200c863419
>   3595	commit	refs/tags/lost+found/e387e14faeea08877c99ddf798d5a4c9f7999c72
>   3763	commit	refs/tags/lost+found/9a1501510c2a842e8597d695fec05563161a84b7
>   3773	commit	refs/tags/lost+found/5cf718038f760d6dc2e21dee3ce98061a975488a
>
> which I was going to delete anyway.  They seem to consist entirely of dummy
> test commits by Glen Morris.

They also contains rewritten history.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: The git mirror is *very* badly screwed up

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> "Eric S. Raymond" <esr@thyrsus.com> writes:
> 
> > Those aren't bzr revisions, they're marks from a git stream dump.
> 
> That's why they are useless.  Tell me the git revisions so I can tell
> you what went wrong.

The parent commit is bcf9acba5a440b6e6d0b9877b23365d3e47765d7

I have visualized the neighborhood of this commit in bubble.png,
which is attached along with an event listing.  These make clear
enough what is going on.

Four commits following :862 - :864, :867, :869, :871 - form a merge
bubble which is closed at :873.  The problem is that something in the 
chain of transmission gave both sides of the bubble the same branch ID,
which should not ever happen.

I think there are several dozen anomalies like this.

> > which I was going to delete anyway.  They seem to consist entirely of dummy
> > test commits by Glen Morris.
> 
> They also contains rewritten history.

History of what?  Can you be clearer about what, if anything, you
think needs to be salvaged from these?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: The git mirror is *very* badly screwed up

[Eric S. Raymond]

Eric S. Raymond <esr@thyrsus.com>:
> Four commits following :862 - :864, :867, :869, :871 - form a merge
> bubble which is closed at :873.  The problem is that something in the 
> chain of transmission gave both sides of the bubble the same branch ID,
> which should not ever happen.

Should not ever happen, that is, during import. It wouldn't be unusual in a
live repo.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: The git mirror is *very* badly screwed up

[Andreas Schwab]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> The parent commit is bcf9acba5a440b6e6d0b9877b23365d3e47765d7

And what's wrong with it?

>> They also contains rewritten history.
>
> History of what?

Of the bzr repository.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: The git mirror is *very* badly screwed up

[Eli Zaretskii]

> Date: Fri, 24 Jan 2014 13:54:30 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: emacs-devel@gnu.org
> 
> Andreas Schwab <schwab@linux-m68k.org>:
> > "Eric S. Raymond" <esr@thyrsus.com> writes:
> > 
> > > Those aren't bzr revisions, they're marks from a git stream dump.
> > 
> > That's why they are useless.  Tell me the git revisions so I can tell
> > you what went wrong.
> 
> The parent commit is bcf9acba5a440b6e6d0b9877b23365d3e47765d7
> 
> I have visualized the neighborhood of this commit in bubble.png,
> which is attached along with an event listing.  These make clear
> enough what is going on.
> 
> Four commits following :862 - :864, :867, :869, :871 - form a merge
> bubble which is closed at :873.  The problem is that something in the 
> chain of transmission gave both sides of the bubble the same branch ID,
> which should not ever happen.

The commits you cited are from the elpa branch.  I don't see why you
should worry about that, since elpa already switched to git anyway.
Its bzr branch is of no use anymore.

Re: The git mirror is *very* badly screwed up

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > Four commits following :862 - :864, :867, :869, :871 - form a merge
> > bubble which is closed at :873.  The problem is that something in the 
> > chain of transmission gave both sides of the bubble the same branch ID,
> > which should not ever happen.
> 
> The commits you cited are from the elpa branch.  I don't see why you
> should worry about that, since elpa already switched to git anyway.
> Its bzr branch is of no use anymore.

Andreas says I should do nothing with that branch.  But if elpa has
its own repo now, probably it should be trimmed out on conversion day.

Somebody, definitely not me and probably Stefan, should make a policy
decision about that. But nobody needs to decide yet.  I have a solid
week of work to do, maybe ten days, maybe even two weeks, cleaning
up other crap in there.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: The git mirror is *very* badly screwed up

[Eli Zaretskii]

> Date: Sat, 25 Jan 2014 01:25:52 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org>:
> > > Four commits following :862 - :864, :867, :869, :871 - form a merge
> > > bubble which is closed at :873.  The problem is that something in the 
> > > chain of transmission gave both sides of the bubble the same branch ID,
> > > which should not ever happen.
> > 
> > The commits you cited are from the elpa branch.  I don't see why you
> > should worry about that, since elpa already switched to git anyway.
> > Its bzr branch is of no use anymore.
> 
> Andreas says I should do nothing with that branch.

Isn't that what I said, too?  (I didn't see Andreas's message until
after writing mine, so you now have 2 independent opinions that say
essentially the same.)

> But if elpa has its own repo now, probably it should be trimmed out
> on conversion day.

What do you mean by "trimmed"?  If you mean removed from the
repository, then I thought you were converting only select branches
anyway, and elpa wasn't one of them.  Bazaar repositories are just
collections of branches, so converting all of them into a git repo is
not necessarily the wisest course of action, to say the least.

> Somebody, definitely not me and probably Stefan, should make a policy
> decision about that. But nobody needs to decide yet.  I have a solid
> week of work to do, maybe ten days, maybe even two weeks, cleaning
> up other crap in there.

I actually don't understand why you are doing a separate new
conversion, instead of relying on what Andreas already did.  AFAIR,
there was no serious discussions of this, and certainly no decision
(except, perhaps, by you).

Goals for repo conversion day

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> I actually don't understand why you are doing a separate new
> conversion, instead of relying on what Andreas already did.  AFAIR,
> there was no serious discussions of this, and certainly no decision
> (except, perhaps, by you).

I am in fact relying on what Andreas already did, not re-lifting the
CVS or Bazaar histories.  But it has some significant problems:

1. Unlifted Bazaar and CVS commit references.

2. CVS commit cliques that should have been coalesced but were not,
   probably because the time window was defaulted too small when parsecvs
   was run.  (Very often these seem to be a pairs of a change and its
   ChangeLog description with an empty comment.)

3. Multiple roots.  Two of the multiples are emacs and elpa, but others 
   are junk lost+found branches which should be carefully inspected and
   then (probably) removed.

4. Obsolete tags (very minor problem, unlike the previous three easily
   fixed in git itself, but I might as well do it while larger cleanups
   are in progress).

5. Unconverted .bzrignores (and possibly .cvsignores) in the history.

Andreas is not to blame for these problems; the tools available to him
were deficient in a number of respects (I had not yet written reposurgeon at
the time of the move to Bazaar). These are all normal issues
which I've seen before in over a dozen large conversions.  

But I want Emacs to have a really high-quality conversion. It is a project
for the ages, one of the great artifacts of hacker culture, and I feel the
history deserves the kind of careful cleaning and restoration one would
give to an Old Master painting.

My quality goals for it include:

(a) Seamless history browsing.  A person looking at the development of
the code should not be distracted by artifacts of VCS changes.  This
means clean, properly unified changesets all the way back, properly
converted ignore files all the way back, and commit references that
are not just opaque cookies left over from previous VCSes.

(b) Information preservation for any future move to another VCS. The
main implication of this goal is not replacing fossil references with
git hashes, as these would present the same problems then that Bazaar
references do now.

(c) Cryptosigning so that future release integrity is protected and
historical release reconstructions can be trusted (that is, assuming
we don't believe the code history has already been corrupted).

(d) Avoidance of metadata representations that are easily stripped or
scrambled if someone gets momentarily forgetful in the future.  This
is why I don't want to rely on lightweight tags or git notes.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Sat, 25 Jan 2014 09:06:38 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> 1. Unlifted Bazaar and CVS commit references.

A mapping file would be more than appropriate for that.  The number of
references is below 200, which is small.  This was suggested already,
and I don't think anyone objected vociferously enough for us to
consider that as a non-option.

And, btw, I'm still not sure you have uncovered all of those
references, since the numbers you posted were significantly smaller
than what I get using simple bzr commands.  So it could be that you
will be investing a non-trivial effort to do a partial job.

> 2. CVS commit cliques that should have been coalesced but were not,
>    probably because the time window was defaulted too small when parsecvs
>    was run.  (Very often these seem to be a pairs of a change and its
>    ChangeLog description with an empty comment.)

I'd say leave that alone.  When Emacs used CVS, it was customary to
commit each file as a separate CVS command (RMS actually asked for
that), and moreover, have the ChangeLog be committed once for several
unrelated changes in the same directory.  So you will be unable to fix
this without a lot of manual work, and for what? we've been living
with this in bzr for several years, with no one complaining.

> 3. Multiple roots.  Two of the multiples are emacs and elpa, but others 
>    are junk lost+found branches which should be carefully inspected and
>    then (probably) removed.

Leave them alone, they don't bother anyone.  Removal can be deferred
for later, if we ever feel it to be necessary.

> 4. Obsolete tags (very minor problem, unlike the previous three easily
>    fixed in git itself, but I might as well do it while larger cleanups
>    are in progress).

Since this is minor, it isn't not worth the trouble, IMO.

> 5. Unconverted .bzrignores (and possibly .cvsignores) in the history.

Why is that a problem?

> Andreas is not to blame for these problems; the tools available to him
> were deficient in a number of respects (I had not yet written reposurgeon at
> the time of the move to Bazaar). These are all normal issues
> which I've seen before in over a dozen large conversions.  

The repository converted by Andreas has one significant advantage: it
is being actively used for many months.  Personally, I trust that more
than any fancy new conversions, especially if we have to switch to git
without a prolonged interregnum, during which both bzr and git are
used (which is probably highly undesirable, if even possible).

> My quality goals for it include:
> 
> (a) Seamless history browsing.  A person looking at the development of
> the code should not be distracted by artifacts of VCS changes.  This
> means clean, properly unified changesets all the way back, properly
> converted ignore files all the way back, and commit references that
> are not just opaque cookies left over from previous VCSes.
> 
> (b) Information preservation for any future move to another VCS. The
> main implication of this goal is not replacing fossil references with
> git hashes, as these would present the same problems then that Bazaar
> references do now.
> 
> (c) Cryptosigning so that future release integrity is protected and
> historical release reconstructions can be trusted (that is, assuming
> we don't believe the code history has already been corrupted).
> 
> (d) Avoidance of metadata representations that are easily stripped or
> scrambled if someone gets momentarily forgetful in the future.  This
> is why I don't want to rely on lightweight tags or git notes.

Noble goals all of them, but I'm skeptical as to whether they can be
achieved in practice.  What's worse, we won't know whether some issues
remained until much later.

So with all due respect, I question the need for this elaborate work,
especially since you evidently know very little about the VCS related
practices around here, certainly during the bzr age.

Comments and opinions by others are welcome.

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Sat, 25 Jan 2014 16:42:11 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> > Date: Sat, 25 Jan 2014 09:06:38 -0500
> > From: "Eric S. Raymond" <esr@thyrsus.com>
> > Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> > 
> > 1. Unlifted Bazaar and CVS commit references.
> 
> A mapping file would be more than appropriate for that.  The number of
> references is below 200, which is small.
                ^^^^^^^^^
Sorry, that number was wrong.  The actual one is slightly above 300,
still very small relative to the 116 thousand revisions we have.

Re: Goals for repo conversion day

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > Date: Sat, 25 Jan 2014 09:06:38 -0500
> > From: "Eric S. Raymond" <esr@thyrsus.com>
> > Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> > 
> > 1. Unlifted Bazaar and CVS commit references.
> 
> A mapping file would be more than appropriate for that.  The number of
> references is below 200, which is small.  This was suggested already,
> and I don't think anyone objected vociferously enough for us to
> consider that as a non-option.

*I* object to this.  On the grounds that I've been through this dance
many times before, and know that such out-of-band representations
generally cost more hassle and deliver less than people expect when
they think them up.
 
> And, btw, I'm still not sure you have uncovered all of those
> references, since the numbers you posted were significantly smaller
> than what I get using simple bzr commands.  So it could be that you
> will be investing a non-trivial effort to do a partial job.

I'm not yet certain I have all the CVS references, but I'm pretty sure
I have all the Bazaar ones now. Your last feedback led me to improve
my search scripts.  I'll do another pass before I'm finished.

> > 2. CVS commit cliques that should have been coalesced but were not,
> >    probably because the time window was defaulted too small when parsecvs
> >    was run.  (Very often these seem to be a pairs of a change and its
> >    ChangeLog description with an empty comment.)
> 
> I'd say leave that alone.  When Emacs used CVS, it was customary to
> commit each file as a separate CVS command (RMS actually asked for
> that), and moreover, have the ChangeLog be committed once for several
> unrelated changes in the same directory.  So you will be unable to fix
> this without a lot of manual work, and for what? we've been living
> with this in bzr for several years, with no one complaining.

Much less manual work than you think.  Reposurgeon was designed with
this kind of fixup in mind.  While it does take some human judgment
to apply the tool properly, the process does not involve grovelling
through every commit by hand.  It's a tolerable amount of effort
even for a repository this large - and if it weren't, I'd add 
capability to reposurgeon until it became so.

As for nobody complaining...this is because your standards are low,
and your standards are low because the tools historically used for these
conversions were mostly shoddy and badly designed. Windows users don't 
complain about Notepad, either, because they don't know any better -
but that doesn't make Emacs a waste of time.

> > 3. Multiple roots.  Two of the multiples are emacs and elpa, but others 
> >    are junk lost+found branches which should be carefully inspected and
> >    then (probably) removed.
> 
> Leave them alone, they don't bother anyone.  Removal can be deferred
> for later, if we ever feel it to be necessary.

See "low standards", above.

> > 4. Obsolete tags (very minor problem, unlike the previous three easily
> >    fixed in git itself, but I might as well do it while larger cleanups
> >    are in progress).
> 
> Since this is minor, it isn't not worth the trouble, IMO.

By itself, no, it wouldn't be.

> > 5. Unconverted .bzrignores (and possibly .cvsignores) in the history.
> 
> Why is that a problem?

See "seamless history browsing".
 
> > Andreas is not to blame for these problems; the tools available to him
> > were deficient in a number of respects (I had not yet written reposurgeon at
> > the time of the move to Bazaar). These are all normal issues
> > which I've seen before in over a dozen large conversions.  
> 
> The repository converted by Andreas has one significant advantage: it
> is being actively used for many months.  Personally, I trust that more
> than any fancy new conversions, especially if we have to switch to git
> without a prolonged interregnum, during which both bzr and git are
> used (which is probably highly undesirable, if even possible).

You appear to have forgotten some important aspects of the plan I
posted.  Andreas's work isn't going to be thrown away, and there isn't
going to be any lengthy interregnum.

The way this is working is that I am building a reposurgeon script that
expresses a sequence of edits to Andreas's mirror. On conversion day 
we will apply that script once, after which everyone can re-clone and
go on as before.

> Noble goals all of them, but I'm skeptical as to whether they can be
> achieved in practice.  What's worse, we won't know whether some issues
> remained until much later.

I know they can be achieved in practice because I have achieved them before,
many times.  Most recently in the conversion of the groff history, but
you could check with the maintainers of NUT or Hercules or robotfindskitten
or Roundup as well. Or the Blender Foundation - blender is a big reposurgeon
conversion done by someone else.

If we find any problems afterwards, I have the tools to fix them. Part of
my commitment is to do that.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Goals for repo conversion day

[Paul Eggert]

Eric S. Raymond wrote:
> If we find any problems afterwards, I have the tools to fix them. Part of
> my commitment is to do that.

Thanks, this part is key. And although I've also suggested shortcuts, if 
you can do the job more completely, so much the better. It'll be nice to 
have a repository that's stainless steel instead of earthenware.

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Sat, 25 Jan 2014 11:01:24 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org>:
> > > Date: Sat, 25 Jan 2014 09:06:38 -0500
> > > From: "Eric S. Raymond" <esr@thyrsus.com>
> > > Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> > > 
> > > 1. Unlifted Bazaar and CVS commit references.
> > 
> > A mapping file would be more than appropriate for that.  The number of
> > references is below 200, which is small.  This was suggested already,
> > and I don't think anyone objected vociferously enough for us to
> > consider that as a non-option.
> 
> *I* object to this.  On the grounds that I've been through this dance
> many times before, and know that such out-of-band representations
> generally cost more hassle and deliver less than people expect when
> they think them up.

With all due respect, this is not necessarily good enough.  You have
come to the project offering help, but no one gave you the right to
make unilateral decisions about these issues.  It won't be you who
will need to use this data in the years to come.  And whatever the
other projects which you converted in the past, I doubt that any of
them had as long and complex history as Emacs.

> > And, btw, I'm still not sure you have uncovered all of those
> > references, since the numbers you posted were significantly smaller
> > than what I get using simple bzr commands.  So it could be that you
> > will be investing a non-trivial effort to do a partial job.
> 
> I'm not yet certain I have all the CVS references, but I'm pretty sure
> I have all the Bazaar ones now. Your last feedback led me to improve
> my search scripts.  I'll do another pass before I'm finished.

I'd appreciate if you posted the final list of the references, when
you are finished with it, so we could have some QA.

> > > 2. CVS commit cliques that should have been coalesced but were not,
> > >    probably because the time window was defaulted too small when parsecvs
> > >    was run.  (Very often these seem to be a pairs of a change and its
> > >    ChangeLog description with an empty comment.)
> > 
> > I'd say leave that alone.  When Emacs used CVS, it was customary to
> > commit each file as a separate CVS command (RMS actually asked for
> > that), and moreover, have the ChangeLog be committed once for several
> > unrelated changes in the same directory.  So you will be unable to fix
> > this without a lot of manual work, and for what? we've been living
> > with this in bzr for several years, with no one complaining.
> 
> Much less manual work than you think.  Reposurgeon was designed with
> this kind of fixup in mind.  While it does take some human judgment
> to apply the tool properly, the process does not involve grovelling
> through every commit by hand.  It's a tolerable amount of effort
> even for a repository this large - and if it weren't, I'd add 
> capability to reposurgeon until it became so.

The problem is not the size of the repository alone.  The problem is
that different portions of a single changeset were committed many
revisions apart.  And I don't even understand (and you didn't explain)
how will you handle the situation I described above, where a single
commit checked in ChangeLog changes for several unrelated commits in
the same directory.  Which commit clique will you assign the ChangeLog
commit to?  The devil is in the details, but you haven't provided any
details about your plans in this matter.  Would you please do that?

> As for nobody complaining...this is because your standards are low,
> and your standards are low because the tools historically used for these
> conversions were mostly shoddy and badly designed. Windows users don't 
> complain about Notepad, either, because they don't know any better -
> but that doesn't make Emacs a waste of time.

Let's agree to keep such low-blow arguments out of this discussion.
They aren't fair, and aren't constructive.  Let's instead assume that
each side knows what they are doing and what they are talking about.

> > > 3. Multiple roots.  Two of the multiples are emacs and elpa, but others 
> > >    are junk lost+found branches which should be carefully inspected and
> > >    then (probably) removed.
> > 
> > Leave them alone, they don't bother anyone.  Removal can be deferred
> > for later, if we ever feel it to be necessary.
> 
> See "low standards", above.

See my response above.

> > > 5. Unconverted .bzrignores (and possibly .cvsignores) in the history.
> > 
> > Why is that a problem?
> 
> See "seamless history browsing".

Sorry, I don't understand.  Please elaborate: what is the relation
between these ignore files and history browsing?

> > The repository converted by Andreas has one significant advantage: it
> > is being actively used for many months.  Personally, I trust that more
> > than any fancy new conversions, especially if we have to switch to git
> > without a prolonged interregnum, during which both bzr and git are
> > used (which is probably highly undesirable, if even possible).
> 
> You appear to have forgotten some important aspects of the plan I
> posted.  Andreas's work isn't going to be thrown away, and there isn't
> going to be any lengthy interregnum.
> 
> The way this is working is that I am building a reposurgeon script that
> expresses a sequence of edits to Andreas's mirror. On conversion day 
> we will apply that script once, after which everyone can re-clone and
> go on as before.

Sorry, I don't see how this changes anything.  You are still going to
make deep changes to the existing mirror.

> > Noble goals all of them, but I'm skeptical as to whether they can be
> > achieved in practice.  What's worse, we won't know whether some issues
> > remained until much later.
> 
> I know they can be achieved in practice because I have achieved them before,
> many times.  Most recently in the conversion of the groff history, but
> you could check with the maintainers of NUT or Hercules or robotfindskitten
> or Roundup as well. Or the Blender Foundation - blender is a big reposurgeon
> conversion done by someone else.

Sorry, been there done that.  The CVS to bzr conversion also seemed
flawless until much later.

> If we find any problems afterwards, I have the tools to fix them. Part of
> my commitment is to do that.

I don't think any of us can in good faith give such promises.

Re: Goals for repo conversion day

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > *I* object to this.  On the grounds that I've been through this dance
> > many times before, and know that such out-of-band representations
> > generally cost more hassle and deliver less than people expect when
> > they think them up.
> 
> With all due respect, this is not necessarily good enough.  You have
> come to the project offering help, but no one gave you the right to
> make unilateral decisions about these issues.  It won't be you who
> will need to use this data in the years to come.  And whatever the
> other projects which you converted in the past, I doubt that any of
> them had as long and complex history as Emacs.

Your doubt is justified.  None of my previous conversions have been 
quite this complex.  The only conversion I have heard of that might have
been hairier was that of Blender, which was done by other people 
using my tools.

But as the size and complexity of the repo goes up, so does the value
of in-band references actually working.  Emacs is an exceptionally *bad*
case for relying solely on an external reference map, not an exceptionally
good one.

> I'd appreciate if you posted the final list of the references, when
> you are finished with it, so we could have some QA.

Here is the current list. It is not final because I expect to resolve
at least a few more  of these, and it is still possible more fossil
references could turn up in odd places.

ChangeLog:
	revno 108687 -> 2012-06-22T21:17:42Z!eggert@cs.ucla.edu
	revno:105007 -> 2011-07-07T04:21:49Z!handa@m17n.org
	r112148 -> 2013-03-26T22:08:58Z!aidalgol@no8wireless.co.nz
	revno:108936 -> 2012-07-07T10:34:37Z!cyd@gnu.org
	revision 106831 -> 2012-01-10T08:27:22Z!cyd@gnu.org
	revision 1.59
	CVS-1.61
	1.61 in CVS
	revno:106608 -> 2011-12-04T17:13:01Z!lekktu@gmail.com
	revno 100789 -> 2010-07-12T05:26:57Z!handa@etlken
	rev. 110325 -> 2012-10-01T18:10:29Z!cyd@gnu.org
	r115470 -> 2013-12-11T19:01:44Z!tzz@lifelogs.com
	of 2012-12-20 (r111276) -> 2012-12-20T11:15:38Z!michael.albinus@gmx.de
	2013-12-11 (r115470) -> 2013-12-11T19:01:44Z!tzz@lifelogs.com
	revno:114543 -> 2013-10-07T01:28:34Z!sdl.web@gmail.com
	revno:113793 -> 2013-08-11T00:07:48Z!lekktu@gmail.com
	revno:113117 -> 2013-06-21T12:24:37Z!lekktu@gmail.com
	r114834 -> 2013-10-29T02:50:24Z!dancol@dancol.org
	revno:113431 -> 2013-07-16T11:41:06Z!jan.h.d@swipnet.se
	revno:113147 -> 2013-06-23T20:29:18Z!lekktu@gmail.com
	revno 101897 -> 2010-10-10T14:43:05Z!dann@ics.uci.edu
	revno 101876 -> 2010-10-09T18:31:12Z!rgm@gnu.org
	revno 100306 -> 2010-05-15T21:21:30Z!raeburn@raeburn.org
	revno 108687 -> 2012-06-22T21:17:42Z!eggert@cs.ucla.edu
	revision 114614 (commit of 2013-10-10) -> 2013-10-10T19:15:33Z!eggert@cs.ucla.edu
	revno:113431 -> 2013-07-16T11:41:06Z!jan.h.d@swipnet.se
	revno 101949 -> 2010-10-13T14:50:06Z!lekktu@gmail.com
	revno:103013 -> 2011-01-28T22:12:05Z!monnier@iro.umontreal.ca
	rev 102609 -> 2010-12-08T08:09:27Z!kfogel@red-bean.com
	revno 101688 -> 2010-09-30T02:53:26Z!lekktu@gmail.com
	revno 101459 -> 2010-09-17T13:30:30Z!monnier@iro.umontreal.ca
	revnos 101381 -> 2010-09-08T14:42:54Z!michael.albinus@gmx.de
	101422 -> 2010-09-13T15:17:01Z!michael.albinus@gmx.de
	rev 100010 -> 2010-04-23T16:26:11Z!monnier@iro.umontreal.ca
	revno:109911 -> 2012-09-07T04:15:56Z!dgutov@yandex.ru
	109621 -> 2012-08-15T03:33:55Z!monnier@iro.umontreal.ca
	revno:88805 -> 2008-06-21T01:38:39Z!monnier@iro.umontreal.ca
	revno:88864 -> 2008-06-22T13:57:28Z!monnier@iro.umontreal.ca
	revno:89810 -> 2008-07-31T05:33:56Z!dann@ics.uci.edu
	revision 106664 -> 2011-12-11T14:49:48Z!vincentb1@users.sourceforge.net
	revno:105285 -> 2011-07-19T15:01:49Z!larsi@gnus.org
	revno:104787 (2011-06-30) -> 2011-06-30T01:09:13Z!larsi@gnus.org
	revno:104988 (2011-07-06) -> 2011-07-06T15:49:19Z!larsi@gnus.org
	revno:101730 (2010-10-02) -> 2010-10-02T13:21:43Z!michael.albinus@gmx.de
	revno:103877 (2011-04-09) -> 2011-04-09T20:28:01Z!cyd@stupidchicken.com
	revno:99634.2.463 (2010-10-09) -> 2010-10-09T04:09:19Z!cyd@stupidchicken.com
	revno:101913 -> 2010-10-11T23:57:49Z!lekktu@gmail.com
	revno 95090 dated 2009-03-06 -> 2009-03-06T07:51:52Z!handa@m17n.org
	revno 101757 -> 2010-10-03T13:59:56Z!dann@ics.uci.edu
	revno 82799 (2007-11-30) -> 2007-11-30T13:57:21Z!jasonr@gnu.org
	2010-07-29 (revno 100939) -> 2010-07-29T16:49:59Z!jan.h.d@swipnet.se
	revno 100928 -> 2010-07-29T03:25:08Z!dann@ics.uci.edu
	revnos 100982 -> 2010-08-05T23:15:24Z!dann@ics.uci.edu
	100984 -> 2010-08-05T23:34:12Z!dann@ics.uci.edu
	revno 99854.1.6 -> 2010-04-17T12:33:05Z!eliz@gnu.org
	revno 99950 -> 2010-04-20T13:31:28Z!eliz@gnu.org
	revno:100708 -> 2010-07-04T07:50:25Z!dann@ics.uci.edu
	revno:110851 -> 2012-11-09T04:10:16Z!monnier@iro.umontreal.ca
	revision 1.1 -> the initial version
	cvs-1.12.1
	Revision 1.694 -> 2004-05-20T23:29:24Z!teirllm@auburn.edu
	revno 108687 -> 2012-06-22T21:17:42Z!eggert@cs.ucla.edu
	revno:108521 -> 2012-06-08T08:44:45Z!eliz@gnu.org
	revno:108341 -> 2012-05-22T16:20:27Z!eggert@cs.ucla.edu
	2011-08-30 (revision 105619) -> 2011-08-30T17:32:44Z!eliz@gnu.org
	2011-08-30 (revision 105619) -> 2011-08-30T17:32:44Z!eliz@gnu.org
	revision 84777 on 2008-02-22 -> 2008-02-22T17:42:09Z!monnier@iro.umontreal.ca
	revno:102982 (2011-01-26) -> 2011-01-26T20:02:07Z!monnier@iro.umontreal.ca
	revision 104625 -> 2011-06-18T18:49:19Z!cyd@stupidchicken.com
	revision 104134 -> 2011-05-06T07:13:19Z!eggert@cs.ucla.edu
	revno:20537 (1998-01-01) -> 1998-01-01T02:27:27Z!rms@gnu.org
	revno:87605 (2008-05-14) -> 2008-05-14T01:40:23Z!handa@m17n.org
	revno:50135 (2003-03-16) -> 2003-03-16T20:45:46Z!storm@cua.dk
	revno:87605 (2008-05-14) -> 2008-05-14T01:40:23Z!handa@m17n.org
	revno:34925 (2000-12-29) -> 2000-12-29T14:24:09Z!gerd@gnu.org
	revno:20537 (1998-01-01) -> 1998-01-01T02:27:27Z!rms@gnu.org
	revno:25013 (1999-07-21) -> 1999-07-21T21:43:52Z!gerd@gnu.org
	revno:43563.1.17 (2002-03-01) -> 2002-03-01T01:17:24Z!handa@m17n.org
	revno:84043 (2008-02-1) -> 2008-02-01T16:01:31Z!miles@gnu.org
	revno:25356 (1999-08-21) -> 1999-08-21T19:30:21Z!gerd@gnu.org
	revno:20870 (1998-02-08) -> 1998-02-08T21:33:56Z!rms@gnu.org
	revno:36704 (2001-03-09) -> 2001-03-09T18:41:50Z!gerd@gnu.org
	revno:32591 (2000-10-17) -> 2000-10-17T16:08:18Z!gerd@gnu.org
	revno:25013 (1999-07-21) -> 1999-07-21T21:43:52Z!gerd@gnu.org
	revno:43563.1.32 (2002-03-01) -> 2002-03-01T01:17:24Z!handa@m17n.org
	revno:14998 (1996-04-12) -> 1996-04-12T06:01:29Z!rms@gnu.org
	revno:86854 (2008-04-19) -> 2008-04-19T19:30:53Z!monnier@iro.umontreal.ca
	revno:20569 (1998-01-02) -> 1998-01-02T21:29:48Z!rms@gnu.org
	revno 103623 -> 2011-03-11T07:24:21Z!eggert@cs.ucla.edu
	revision 1.32 of saveplace.el -> saveplace.el at 2005-05-29T08:36:26Z!rms@gnu.org
	revision 1.30  of saveplace.el -> saveplace.el at 2005-04-10T23:32:00Z!rms@gnu.org
	version 1.100 -> 2007-12-06T19:56:41Z!deego@gnufans.org
	erc.el 1.39 -> 2007-12-01T03:41:01Z!rgm@gnu.org
	revision 1.104, made on 2000-05-21 -> 2000-05-21T17:04:47Z!fx@gnu.org
	2007-07-18 (revision 1.51)
	revision 1.90 (commitid mWoPbju3pgNotDps) -> 2007-07-13T18:16:17Z!kfogel@red-bean.com
	revision 1.117 -> 2008-10-29T17:42:49Z!cyd@stupidchicken.com
	1.85
	1.878
	1.113
	1.244
	1.34
	1.233
	rev 1.82 -> 1994-08-03T07:39:00Z!rms@gnu.org
	1.70 (Jan 5 changes) -> 1994-01-03T07:21:12Z!rms@gnu.org
	r99212 -> 2009-12-29T07:22:00Z!nickrob@snap.net.nz
	rev. 110325 -> 2012-10-01T18:10:29Z!cyd@gnu.org
	revno r112320 -> 2013-04-18T00:12:33Z!monnier@iro.umontreal.ca

Change comments:
	bzrs 111300 -> 2012-12-22T19:57:35Z!rgm@gnu.org
	111840 -> 2013-02-21T02:42:30Z!eggert@cs.ucla.edu
	revision 111647 -> 2013-02-01T07:23:18Z!dmantipov@yandex.ru
	revno:11026 -> 1995-03-15T21:55:37Z!kwzh@gnu.org
	revno:88864 -> 2008-06-22T13:57:28Z!monnier@iro.umontreal.ca
	revno:88805 -> 2008-06-21T01:38:39Z!monnier@iro.umontreal.ca
	revno:89810 -> 2008-07-31T05:33:56Z!dann@ics.uci.edu
	revision 10835 -> 1995-02-25T20:57:45Z!rms@gnu.org
	revision 106726 -> 2011-12-23T14:51:51Z!eliz@gnu.org
	revision 87208 -> 2008-05-02T07:12:59Z!esr@snark.thyrsus.com
	revision 84777 on 2008-02-22 -> 2008-02-22T17:42:09Z!monnier@iro.umontreal.ca
	revno:99634.2.463 (2010-10-09) -> 2010-10-09T04:09:19Z!cyd@stupidchicken.com
	revno:101913 (2010-10-12). -> 2010-10-11T23:57:49Z!lekktu@gmail.com
	revno:20537 (1998-01-01) -> 1998-01-01T02:27:27Z!rms@gnu.org
	revno:87605 (2008-05-14) -> 2008-05-14T01:40:23Z!handa@m17n.org
	revno:87605 (2008-05-14) -> 2008-05-14T01:40:23Z!handa@m17n.org
	revno:34925 (2000-12-29) -> 2000-12-29T14:24:09Z!gerd@gnu.org
	revno:20537 (1998-01-01) -> 1998-01-01T02:27:27Z!rms@gnu.org
	revno:25013 (1999-07-21) -> 1999-07-21T21:43:52Z!gerd@gnu.org
	revno:43563.1.16 (2002-03-01) -> 2002-03-01T01:16:34Z!handa@m17n.org
	revno:84043 (2008-02-1) -> 2008-02-01T16:01:31Z!miles@gnu.org
	revno:20870 (1998-02-08) -> 1998-02-08T21:33:56Z!rms@gnu.org
	revno:36704 (2001-03-09) -> 2001-03-09T18:41:50Z!gerd@gnu.org
	revno:32591 (2000-10-17) -> 2000-10-17T16:08:18Z!gerd@gnu.org
	revno:25356 (1999-08-21) -> 1999-08-21T19:30:21Z!gerd@gnu.org
	revno:14998 (1996-04-12) -> 1996-04-12T06:01:29Z!rms@gnu.org
	revno:86854 (2008-04-19) -> 2008-04-19T19:30:53Z!monnier@iro.umontreal.ca
	revno:20569 (1998-01-02) -> 1998-01-02T21:29:48Z!rms@gnu.org
	r100577 -> 2010-06-10T12:56:11Z!michael.albinus@gmx.de
	CVS rev 1.49, 2001-09-12
	CVS rev 1.47, 2003/01/27
	CVS r1.35
	revno 95090 dated 2009-03-06 -> 2009-03-06T07:51:52Z!handa@m17n.org
	2005-02-15 (revno 60055) -> 2005-02-15T23:19:26Z!jasonr@gnu.org
	r111320 -> 2012-12-24T15:56:17Z!eliz@gnu.org
	revno 99854.1.6 -> 2010-04-17T12:33:05Z!eliz@gnu.org
	revno 99950 -> 2010-04-20T13:31:28Z!eliz@gnu.org
	revision 99649 -> 2010-03-12T16:34:27Z!eliz@gnu.org
	rev 99649 -> 2010-03-12T16:34:27Z!eliz@gnu.org
	rev 99553 -> 2010-02-24T22:07:26Z!bob@gnu.org
	revno 99212 -> 2009-12-29T07:22:00Z!nickrob@snap.net.nz
	revision 94343 -> 2009-01-30T13:06:07Z!lekktu@gmail.com
	revision 1.32 -> 2005-05-29T08:36:26Z!rms@gnu.org
	revision 1.30 -> 2005-04-10T23:32:00Z!rms@gnu.org
	version 1.100 -> 2007-12-06T19:56:41Z!deego@gnufans.org
	r1.135 -> 2009-10-10T21:48:22Z!kfogel@red-bean.com
	rev 1.114
	1.878
	revision 1.117 -> 2008-10-29T17:42:49Z!cyd@stupidchicken.com
	rev 1.14395
	revision 1.56
	3.85
	1.17
	revision 1.69
	revision 1.1 -> initial revision
	rev 1.5
	revisions 1.40
	1.41
	1.39-> 2007-12-01T03:41:01Z!rgm@gnu.org
	revision 1.104
	revision 1.51
	revision 1.90 (commitid mWoPbju3pgNotDps) -> 2007-07-13T18:16:17Z!kfogel@red-bean.com
	revision 1.1509
	revision 7.8
	CVS v1.12.8 and 1.12.9
	cvs-1.12.1
	1.103
	HEAD (1.72)
	v1.275
	1.58
	v1.5046
	v1.5039
	rev 1.82 -> 1994-08-03T07:39:00Z!rms@gnu.org
	rev. 1.761
	revision 1.3831
	1.3832
	revision 1.12
	revision 1.13
	revision 1.14
	revision 1.15

The ChangeLog references are not attributed to individual files
because they moved as the files rotated.

Some of the remaining CVS references cannot be reseolved within the Emacs
history; they actually point to other projects.  One particularly fertile
source of these, which I think accounts for this group

	1.85
	1.878
	1.113
	1.244
	1.34
	1.233

in ChangeLogs, is the CVS history of the erc files before they were merged
into Emacs.

> The problem is not the size of the repository alone.  The problem is
> that different portions of a single changeset were committed many
> revisions apart.  And I don't even understand (and you didn't explain)
> how will you handle the situation I described above, where a single
> commit checked in ChangeLog changes for several unrelated commits in
> the same directory.  Which commit clique will you assign the ChangeLog
> commit to?  The devil is in the details, but you haven't provided any
> details about your plans in this matter.  Would you please do that?

I see we are using the term "changeset" slightly differently, and this has
produced some confusion.

The uncoalesced changesets I am looking for are not defined by "all
share the same ChangeLog entry" (though usually that is the case).
You are quite right that attempting to coalesce all of those would
produce perverse results in cases of several unrelated commits.

Fortunately, most of the unresolved cliques are not like this.  The
usual case, in this conversion as in others I've seen (such as groff)
is that an unresolved clique consists of one or several closely
related changes and one ChangeLog modification, without intervening
commits by others.  This is what I think of as a changeset.

Normally tools such as parsecvs collect these into single changesets.  
But these converters have a maximum coalescence window.  If such a span
of commits took place over a longer period of time than the window, it
won't be coalesced. 

The problem is that the default time windows on these converters are
set small in order to avoid false-positive matches.  Experience has
taught me that this is a mostly imaginary problem; the window would
have been better set to infinity in almost every case I have seen.

The result of a too-small commit window is that some genuine changesets
(not the edge case you are pointing at) do not get coalesced. In your
edge case, the least bad thing to do is accept that the ChangeLog entry
must remain its own changeset; sometimes you can get partial 
coalescence in the file changes.

When there is CVS in the history, a standard part of my cleanup is
basically to run a coalescence pass with a very long window.
Semi-automating this operation, so it (a) doesn't have to be done
manually, but (b) is easily checked by skilled human judgment, was
one of the purposes for which I originally wrote reposurgeon.

Fortunately the bad cases aren't actually very common.

> > > > 5. Unconverted .bzrignores (and possibly .cvsignores) in the history.
> > > 
> > > Why is that a problem?
> > 
> > See "seamless history browsing".
> 
> Sorry, I don't understand.  Please elaborate: what is the relation
> between these ignore files and history browsing?

In a properly done conversion, file ignores don't abruptly stop working
bevcause you browsed back past the point of conversion and what should
be .gitignore files are nmow .bzrignores or .cvsignores.

> > The way this is working is that I am building a reposurgeon script that
> > expresses a sequence of edits to Andreas's mirror. On conversion day 
> > we will apply that script once, after which everyone can re-clone and
> > go on as before.
> 
> Sorry, I don't see how this changes anything.  You are still going to
> make deep changes to the existing mirror.

Yes, for arguable values of "deep". As Paul Eggert (I think) said, I'm
after a result that is stainless steel rather than earthenware. With
ugly cracks in it.

> > > Noble goals all of them, but I'm skeptical as to whether they can be
> > > achieved in practice.  What's worse, we won't know whether some issues
> > > remained until much later.
> > 
> > I know they can be achieved in practice because I have achieved them before,
> > many times.  Most recently in the conversion of the groff history, but
> > you could check with the maintainers of NUT or Hercules or robotfindskitten
> > or Roundup as well. Or the Blender Foundation - blender is a big reposurgeon
> > conversion done by someone else.
> 
> Sorry, been there done that.  The CVS to bzr conversion also seemed
> flawless until much later.

There are several differences this time.  One of the most important is that
the state of the art has advanced.  My tools do things that would have been
impossible or impractical before they existed.  I have auditing capabilities
you would probably have to work a bit to even imagine.

As a relatively trivial example - if Stefan or some other person with
policy authority makes the call, I could reliably split elpa out into
its own repo with one short command in the reposurgeon DSL.
 
> > If we find any problems afterwards, I have the tools to fix them. Part of
> > my commitment is to do that.
> 
> I don't think any of us can in good faith give such promises.

The span of my contributions to Emacs is measures in decades.  I do not 
think you need to fear that I will vanish before this job is done.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Sat, 25 Jan 2014 16:01:32 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> But as the size and complexity of the repo goes up, so does the value
> of in-band references actually working.  Emacs is an exceptionally *bad*
> case for relying solely on an external reference map, not an exceptionally
> good one.

There's no argument about the higher value of having all the
references resolved.  What I fear of is the inordinate amount of work
that might require, for too little benefit, and the unintended
consequences of a too-deep surgery on the history that will be needed.
Already the effort to get the list of references right devoured many
messages (and I'm sure each message caused a non-trivial amount of
work), and we are still not there (see below).  In particular, it
worries me that you seem to be unable to extract the full list of bzr
revno references, after so many attempts.  Why this doesn't worry you,
and why you still refuse to accept that maybe, just maybe, this is a
lot of effort for a relatively small gain, is beyond me.  If this is
in any way indicative of the other problematic issues of the
conversion, then "Houston, we have a problem", indeed.

> > I'd appreciate if you posted the final list of the references, when
> > you are finished with it, so we could have some QA.
> 
> Here is the current list. It is not final because I expect to resolve
> at least a few more  of these, and it is still possible more fossil
> references could turn up in odd places.

> ChangeLog:
> [...]

I found that at least these ones are missing:

  lisp/ChangeLog.15 references 103083
  lisp/ChangeLog.16 references 103471 and 107149
  src/ChangeLog.12 references 104015 and 103913

> Change comments:
> [...]

This list of 40 references in the commit messages to bzr revisions is
definitely incomplete.  It misses many references (I counted more than
300 overall, including those you show).  Here are just a few that you
missed, and only from the trunk branch:

  r116131 references 116113
  r116056 references 116055
  r115997 references 115992
  r115964 references 115961
  r115920 references 115918
  r115859 references 115838
  r115029 references 112851
  r114978 references 114965
  r114798 references 114795
  r112011 references 112010
  r106733.1.27 references 111919
  r110764.1.510 references 111040
  r110764.1.338 references 111367 and 111368
  r110879 references 110857 (from emacs-24 branch)
  r110306 references 110305
  r99375 references 99362

It sounds like the scripts or methods you are using to find such
references are not catching some of them.  E.g., bare numbers, without
any leading "r" or "revno:" etc. are mostly (or maybe completely)
missing.

Given this quality, I once again question the need for all this work.
If we cannot guarantee coverage very close to 100%, what would be the
value of such a partial conversion?  More importantly, do we have
reasonably effective methods of QA for the results?  The omissions I
discovered are based on simple bzr commands followed by manual
inspection (to avoid quite a few false positives); unless we can come
up with better ways that don't involve manual labor, the overall
quality will not be high enough, as manual labor is inherently error
prone.

Btw, what about references to repositories of other projects?  Here's
one example (from trunk):

    revno: 110764.1.388
    committer: Bastien Guerry <bzg@gnu.org>
    branch nick: emacs-24
    timestamp: Tue 2013-01-08 19:49:37 +0100
    message:
      Merge Org up to commit 4cac75153.  Some ChangeLog formatting fixes.

Are we going to replace the git sha1 here by something more universal?
If so, there's much more work around the corner; if not, why does it
make sense to insist on doing that for Emacs's own branches?

> Some of the remaining CVS references cannot be reseolved within the Emacs
> history; they actually point to other projects.  One particularly fertile
> source of these, which I think accounts for this group
> 
>         1.85
>         1.878
>         1.113
>         1.244
>         1.34
>         1.233
> 
> in ChangeLogs, is the CVS history of the erc files before they were merged
> into Emacs.

See above: this is just the tip of the iceberg.  I think you will find
much more of such references, with Org, CEDET, MH-E, and Gnus being
the most frequent ones.  Doesn't leaving those out of this conversion
undermine the goal?

> > The problem is not the size of the repository alone.  The problem is
> > that different portions of a single changeset were committed many
> > revisions apart.  And I don't even understand (and you didn't explain)
> > how will you handle the situation I described above, where a single
> > commit checked in ChangeLog changes for several unrelated commits in
> > the same directory.  Which commit clique will you assign the ChangeLog
> > commit to?  The devil is in the details, but you haven't provided any
> > details about your plans in this matter.  Would you please do that?
> 
> I see we are using the term "changeset" slightly differently, and this has
> produced some confusion.
> 
> The uncoalesced changesets I am looking for are not defined by "all
> share the same ChangeLog entry" (though usually that is the case).
> You are quite right that attempting to coalesce all of those would
> produce perverse results in cases of several unrelated commits.
> 
> Fortunately, most of the unresolved cliques are not like this.  The
> usual case, in this conversion as in others I've seen (such as groff)
> is that an unresolved clique consists of one or several closely
> related changes and one ChangeLog modification, without intervening
> commits by others.  This is what I think of as a changeset.

I thought a "changeset" was well defined in the context of a VCS.  My
definition is a set of changes made as part of working on a single
isolated issue.  IOW, what would have constituted a single indivisible
commit with our current procedures.

Your definition sounds subtly different, and you didn't define
"closely related changes", so it's hard to judge its exact meaning.
As for "one ChangeLog modification" and "without intervening commits",
see below.

> Normally tools such as parsecvs collect these into single changesets.  
> But these converters have a maximum coalescence window.  If such a span
> of commits took place over a longer period of time than the window, it
> won't be coalesced. 

From a cursory look I had at the current git mirror, no coalescing was
done there.  But perhaps I'm missing something; Andreas, can you
please comment on this?

> When there is CVS in the history, a standard part of my cleanup is
> basically to run a coalescence pass with a very long window.
> Semi-automating this operation, so it (a) doesn't have to be done
> manually, but (b) is easily checked by skilled human judgment, was
> one of the purposes for which I originally wrote reposurgeon.
> 
> Fortunately the bad cases aren't actually very common.

Can we take a real-life use case, please?  Please show the cliques
produced by your analysis in this range of bzr revisions on the trunk:
39997..40058.  You can see the details with these bzr commands:

 . This will show a 1-line summary for every revision in the range:

     bzr log --line -r39997..40058

 . This will show the full commit messages and other meta-data of a
   single revision, 40000 in the example (can also be used with a
   range -rNNN..MMM):

     bzr log --long --show-ids -c40000

 . This will show the files modified/added/deleted by a single
   revision (can also be used with a range -rNNN..MMM):

     bzr status -c40000

The above range of revisions shows a typical routine of commits when
Emacs was using CVS; in particular, "*** empty log message ***" are
most probably ChangeLog commits which usually followed commits of the
files whose log entries are in the ChangeLog change.  Note that the
commit messages are almost always different (they are actually the
ChangeLog entries for the files being committed), although the changes
belong to the same changeset.  Also note how commits by different
people working on separate changesets sometimes overlap, as in
revisions 40033..40038.

How will these be handled during your proposed conversion?  And what
will be the commit messages of the coalesced commits?

> > > > > 5. Unconverted .bzrignores (and possibly .cvsignores) in the history.
> > > > 
> > > > Why is that a problem?
> > > 
> > > See "seamless history browsing".
> > 
> > Sorry, I don't understand.  Please elaborate: what is the relation
> > between these ignore files and history browsing?
> 
> In a properly done conversion, file ignores don't abruptly stop working
> bevcause you browsed back past the point of conversion and what should
> be .gitignore files are nmow .bzrignores or .cvsignores.

So you will be adding .gitignore to revisions where there was none?
If not, how do you plan on attacking this issue?

> > > The way this is working is that I am building a reposurgeon script that
> > > expresses a sequence of edits to Andreas's mirror. On conversion day 
> > > we will apply that script once, after which everyone can re-clone and
> > > go on as before.
> > 
> > Sorry, I don't see how this changes anything.  You are still going to
> > make deep changes to the existing mirror.
> 
> Yes, for arguable values of "deep". As Paul Eggert (I think) said, I'm
> after a result that is stainless steel rather than earthenware. With
> ugly cracks in it.

I have my doubts about the "stainless steel" part, sorry.
Unfortunately, nothing you've said so far contributes to my confidence
in the outcome.  And if the outcome will more like "earthenware" than
"stainless steel", then we might as well continue using what we have
now in the existing mirror.

> > > > Noble goals all of them, but I'm skeptical as to whether they can be
> > > > achieved in practice.  What's worse, we won't know whether some issues
> > > > remained until much later.
> > > 
> > > I know they can be achieved in practice because I have achieved them before,
> > > many times.  Most recently in the conversion of the groff history, but
> > > you could check with the maintainers of NUT or Hercules or robotfindskitten
> > > or Roundup as well. Or the Blender Foundation - blender is a big reposurgeon
> > > conversion done by someone else.
> > 
> > Sorry, been there done that.  The CVS to bzr conversion also seemed
> > flawless until much later.
> 
> There are several differences this time.  One of the most important is that
> the state of the art has advanced.  My tools do things that would have been
> impossible or impractical before they existed.  I have auditing capabilities
> you would probably have to work a bit to even imagine.

The important question here is "is your best good enough?"  I have
absolutely no idea what is the answer to that question, and frankly,
your way of promoting your tools and techniques doesn't help at all.
Neither do the apparent deficiencies in identifying revision
references shown above.

If you really want to build confidence in your methods and tools, some
kind of statistics about the conversion jobs done using them, and the
time passed since the conversion would probably be a good start.
(Yes, time since conversion is important because the problems are
usually subtle and don't stick out until much later.)  Detailed
description of the planned steps during the conversion and how you
intend to control the quality of each step, will also be appreciated.

> As a relatively trivial example - if Stefan or some other person with
> policy authority makes the call, I could reliably split elpa out into
> its own repo with one short command in the reposurgeon DSL.

This is great, but doesn't really address the worrisome aspects of the
conversion we care about.  We no longer care about the elpa branch in
the bzr repository.  We do care about the few other branches, such as
emacs-24.  And it is not even clear what will become of those after
the conversion; the reposurgeon man page cites a limitation related to
that, allegedly stemming from some (imaginary) bzr confusion between
branches and repositories, but ends up saying nothing about the
branches after the conversion.  Will they end up in a single git
repository, like any other git branches, or won't they?  Will the
merges between those branches show up as expected in git DAG?  How
will merges from external branches (such as Org or MH-E) or from local
feature branches be represented?  Those are much more important issues
than the ability to split elpa.

> > > If we find any problems afterwards, I have the tools to fix them. Part of
> > > my commitment is to do that.
> > 
> > I don't think any of us can in good faith give such promises.
> 
> The span of my contributions to Emacs is measures in decades.  I do not 
> think you need to fear that I will vanish before this job is done.

I was talking about the "problems afterwards" part.  I don't question
your intentions, but life is not an entirely predictable endeavor.
Perhaps you have a way to tell the future, but in that case, I may
wish to hire you to help me with my stock investments.

Re: Goals for repo conversion day

[Eric S. Raymond]

[-- Attachment #1: Type: text/plain, Size: 17898 bytes --]

Eli Zaretskii <eliz@gnu.org>:
>                                       Why this doesn't worry you,
> and why you still refuse to accept that maybe, just maybe, this is a
> lot of effort for a relatively small gain, is beyond me.  If this is
> in any way indicative of the other problematic issues of the
> conversion, then "Houston, we have a problem", indeed.

What I refuse to accept is doing a job that is below my standards of
quality, if I'm going to do it at all.  You cannot argue me out of
that by telling me it's too much work, because I simply don't accept
that as a valid reason to settle for slipshod results.  Instead, I
upgrade my tools.

> I found that at least these ones are missing:
> 
>   lisp/ChangeLog.15 references 103083
>   lisp/ChangeLog.16 references 103471 and 107149
>   src/ChangeLog.12 references 104015 and 103913

Thank you for finding these.  This is a useful bug report.

To illustrate my methods, I fixed this by adding those revnos to the 
ChangeLog section of the map file I enclosed in my last mail (it is
the file FOSSILS in my conversion directory).  Then I ran a Python
script called 'decorate.py' that patched in the corresponding action
stamps.  The point is that I didn't have to do the lookup by hand; the
fixup took less time to do than to describe.

The map that decorate.py uses is in turn generated by a second script,
bzrlog2map, that filters the putput of bzr log --levels 0 into an
association between revnos and action stamps.  Here are the first
few lines:

116082	2014-01-20T16:55:28Z!eggert@cs.ucla.edu
116081	2014-01-20T16:47:41Z!eggert@cs.ucla.edu
116080	2014-01-20T08:52:44Z!juri@jurta.org
116079	2014-01-20T08:45:56Z!juri@jurta.org
116078	2014-01-20T08:15:16Z!eggert@cs.ucla.edu
116077	2014-01-20T07:56:28Z!eggert@cs.ucla.edu
116076	2014-01-20T01:21:18Z!rgm@gnu.org
116075	2014-01-20T00:54:19Z!rgm@gnu.org
116074	2014-01-19T16:59:51Z!rudalics@gmx.at
116073	2014-01-19T15:42:48Z!eliz@gnu.org
116072	2014-01-19T13:28:16Z!handa@gnu.org
115426.2.11	2014-01-19T13:27:34Z!handa@gnu.org
115426.2.10	2014-01-19T13:26:21Z!handa@gnu.org
115426.2.9	2014-01-19T12:42:37Z!handa@gnu.org
115426.2.8	2014-01-18T00:24:14Z!handa@gnu.org
115426.2.7	2014-01-18T00:24:03Z!handa@gnu.org

This is MAP in my conversion directory; I rebuild it occasionally
to be sure new revs are included,

The point of having two maps rather than one is this: at some point
I'm going to mechanically compile FOSSILS into a list of reposurgeon
commands.  For example, this:

ChangeLog:
	revno 108687 -> 2012-06-22T21:17:42Z!eggert@cs.ucla.edu

will become something like this 

=B & [ChangeLog] filter --replace /\brevno 108687\b/2012-06-22T21:17:42Z!eggert@cs.ucla.edu/

That command translates into English as: Over the set of all blobs in
the history with paths containing the string 'ChangeLog', replace
'revno 108687' (preceded and followed by breaking characters) by its
corresponding action stamp.

I could, in theory, generate a humongous and guaranteed-exhaustive set
of these commands directly from MAP. If I did that, though, the
conversion day script might would many hours to run, most of that
spent on generated commands that are no-ops. There could also be
unhappiness related to revision numbers short enough to false-match
numeric tokens that are nothing of the kind.

Instead, FOSSILS both drives and documents the minimum set of changes
required.  The cost is that I have to maintain the list of source
tokens to be replaced partly by hand.  This is normal and acceptable;
I often deal with similar issues in Subversion repositories.

> It sounds like the scripts or methods you are using to find such
> references are not catching some of them.  E.g., bare numbers, without
> any leading "r" or "revno:" etc. are mostly (or maybe completely)
> missing.

Looking at bzrlog2map, I see you're right.  One of my to-do items
was to add to it a scanner that would turn up likely reference-string
candidates.  I forgot I hadn't actually done that yet.

> Given this quality, I once again question the need for all this work.

That is incoherent.  Whether the work is needed has *nothing* to do with
whether it is well implemented yet.

> If we cannot guarantee coverage very close to 100%, what would be the
> value of such a partial conversion?

Exactly proportional to the coverage, of course.  Every single
reference that is easily chased by human eyeball or indexing tool
(e.g. *not* a cookie that is meaningless because its context is gone)
increases the utility of the conversion.  Complete transparency of
reference is best; more is better than less; partial is better than
none.

The history is too messy for us to get 100% coverage (too many
external CVS references), but that is not an argument that we should
settle for zero.

>                                    More importantly, do we have
> reasonably effective methods of QA for the results?  The omissions I
> discovered are based on simple bzr commands followed by manual
> inspection (to avoid quite a few false positives); unless we can come
> up with better ways that don't involve manual labor, the overall
> quality will not be high enough, as manual labor is inherently error
> prone.

This is why I explained my workflow.  Once a reference has been identified
and put in FOSSILS, none of the remaining steps are vulnerable to human
error.  (My scripts could have bugs, of course.  But they're not very
complex, so we can have reasonably high confidence in them.)

> Btw, what about references to repositories of other projects?  Here's
> one example (from trunk):
> 
>     revno: 110764.1.388
>     committer: Bastien Guerry <bzg@gnu.org>
>     branch nick: emacs-24
>     timestamp: Tue 2013-01-08 19:49:37 +0100
>     message:
>       Merge Org up to commit 4cac75153.  Some ChangeLog formatting fixes.
> 
> Are we going to replace the git sha1 here by something more universal?

No, because there is no notation and no resolution protocol for such
references.  If there were such a thing, I would be right on top of using it.  
Actually, if there were such a thing, it would more than likely have been my
invention to begin with...

> If so, there's much more work around the corner; if not, why does it
> make sense to insist on doing that for Emacs's own branches?

Because that *can* be done, and every successful internalization adds utility
by (a) removing an impediment to browsing and (b) documenting a causal link.

> See above: this is just the tip of the iceberg.  I think you will find
> much more of such references, with Org, CEDET, MH-E, and Gnus being
> the most frequent ones.  Doesn't leaving those out of this conversion
> undermine the goal?

Yes, of course it does.  Don't let the unachievable perfect be the enemy of the 
achievable good!

(Damn, now you've started me thinking about prefixing action stamps
with name lookups to a registry of repositories.  If I invent a
practical solution to this it's going to be partly your fault...)

> I thought a "changeset" was well defined in the context of a VCS.

In modern VCSes like Bazaar, hg, and git, yes, it is a well-defined
concept.  This conversion creates confusing cases for two reasons.  One
is the vagaries of CVS; the other is ChangeLog entries, which carry
some of the semantic freight of VCS changesets without having the
atomicity and time-locality properties that they automatically have when
the VCS actually implements them.

The result is that one Emacs/Zaretskii "changeset" usually corresponds
to one modern VCS changeset, but not always. When the correspondence breaks
down, one Emacs/Zaretskii "changeset" maps to two or more VCS changesets,
one of which is likely to be a Changelog entry that is semantically
bound to the others but a singleton changeset that the VCS doesn't know
is connected to them.

> My definition is a set of changes made as part of working on a single
> isolated issue.  IOW, what would have constituted a single indivisible
> commit with our current procedures.

The Bazaar portion of the history isn't the problem, the CVS part is.
There are many instances in the CVS part of the Emacs history that
look something like this:

1. Eli changes file A and commits it
2. Eli changes file B and commits it with an identical change comment.
3. Eric changes file C and commits it
4. Eli commits a ChangeLog entry describing the A and B changes
5. Eric commits a ChangeLog entry describing the C changes

In your terms, there are two changesets here: {1,2,4} and {3,5).
But when parsecvs runs, the result will probably look like this:

Changeset 1 - {1,2}
Changeset 2 - {3}
Changeset 3 - {4}
Changeset 4 - {5}

Changesets 1 and 3 don't get joined because the intervening commit 
prevented parsecvs from recognizing that they should be coalesced.

(Actually the behavior is a little better than this: parsecvs did
coalescence by branch, so if commit 3 is on a different branch than
1 and 2 the right thing will happen.)

Here's where the vagaries of CVS come in. For various stupid random
CVS-is-brain-damaged reasons there may have been enough skew between
the recorded commit times of 1 and 2 that *they* don't get coalesced,
even though that's what notional-Eli intended.

*That* kind of defect (eligible commits that didn't fit inside too
small a time window) is what reposurgeon was originally designed to 
fix.  These are very, *very* common in crappy CVS lifts, and reposurgeon
can fix them automatically.

There is another case common in the Emacs history that can be
coalesced.  That is: a file modification immediately followed by a
ChangeLog change describing it - but with an empty change comment on
the ChangeLog change, which parcecvs refuses to consider matching to
anything else.  These do have to be fixed up by hand.  I haven't tried
yet.

> From a cursory look I had at the current git mirror, no coalescing was
> done there.  But perhaps I'm missing something; Andreas, can you
> please comment on this?

Look for commits that predate the Bazaar transition but change multiple
files. You'll find parsecvs made those.

> Can we take a real-life use case, please?  Please show the cliques
> produced by your analysis in this range of bzr revisions on the trunk:
> 39997..40058.  You can see the details with these bzr commands:
> 
>  . This will show a 1-line summary for every revision in the range:
> 
>      bzr log --line -r39997..40058
> 
>  . This will show the full commit messages and other meta-data of a
>    single revision, 40000 in the example (can also be used with a
>    range -rNNN..MMM):
> 
>      bzr log --long --show-ids -c40000
> 
>  . This will show the files modified/added/deleted by a single
>    revision (can also be used with a range -rNNN..MMM):
> 
>      bzr status -c40000
> 
> The above range of revisions shows a typical routine of commits when
> Emacs was using CVS; in particular, "*** empty log message ***" are
> most probably ChangeLog commits which usually followed commits of the
> files whose log entries are in the ChangeLog change.  Note that the
> commit messages are almost always different (they are actually the
> ChangeLog entries for the files being committed), although the changes
> belong to the same changeset.  Also note how commits by different
> people working on separate changesets sometimes overlap, as in
> revisions 40033..40038.
> 
> How will these be handled during your proposed conversion?  And what
> will be the commit messages of the coalesced commits?

I think the example I showed above explains most of this.  I'd have to grovel
through all the timestamps to find out if automatic coalescence would catch
any of the cliques in your span, but I can say that (for example) this:

40050: Miles Bader 2001-10-19 *** empty log message ***
40049: Miles Bader 2001-10-19 Exit if we can't find some variable.

looks like something the "lint" command in reposurgeon would catch. I
would then eyeball it to check that 40050 is the changelog tweak 
describing 40049 and write something like this into the lift script:

<40049>..<40050> squash --pushback

The effect would be to merge 40050's Changelog fileop into 40049, which
would keep its comment.  The children and parents of the sequashed
commit would be what you think.

And yes, <40049> would be a legal commit reference in reposurgeon.  Provided
I did this first:

read fossils <MAP

which is the other use of the MAP file I described previously.

> > In a properly done conversion, file ignores don't abruptly stop working
> > bevcause you browsed back past the point of conversion and what should
> > be .gitignore files are nmow .bzrignores or .cvsignores.
> 
> So you will be adding .gitignore to revisions where there was none?
> If not, how do you plan on attacking this issue?

By converting .bzrignore files in place to .gitignores.

> If you really want to build confidence in your methods and tools, some
> kind of statistics about the conversion jobs done using them, and the
> time passed since the conversion would probably be a good start.

I can tell you the most important statistics.  For three years of 
doing conversions on projects including GPSD, NUT, Hercules, Roundup,
Battle For Wesnoth, robotfindskitten, groff, and several others,
I can tell you three numbers:

1. Time passed since conversion: tops out at 3 years for GPSD, about 2
years each for NUT and Hercules.

2. Number of defects I found myself after delivering a final
conversion: three. (All in Battle For Wesnoth. Two CVS usernames
didn't get properly mapped to git-style IDs because the attribution
file I was using at conversion time was incomplete.)

3. Number of defects subsequently reported by project dev groups:
zero.  Yes, *zero*.

One of the dev groups (Roundup, for which I did SVN->git) later moved
to hg for political reasons.  Otherwise those repositories are still
in active use by multiple developers, and have been for a cumulative
hundreds of thousands of hours.

I won't represent that I think none of my finished conversions has
ever had an error; that would be highly unlikely.  What is true is
that any errors they had were so minor that nobody has thought
it was worth bugging me about them.

As a matter of history, GPSD and Hercules were early test conversions.
NUT (Network UPS tools) was reposurgeon's trial by fire; I went into
that with a usable beta-grade tool, came out of it with something good
enough that the much bigger and nastier Blender conversion could be
done by *people who weren't me*.

By the time I did groff, late last year, my tools and procedures for
normal cases were pretty well routinized and bulletproofed.  You can
read about them here:

DVCS Migration HOWTO: http://www.catb.org/esr/dvcs-migration-guide.html

There's even a makefile that semi-automates the conversion steps.

That said, Emacs is a bit abnormal.  The kind of case I'm used to
handling is Subversion repo with a fossil layer of CVS, having on the
close order of a decade of history and a commit count in the 3K-30K
range (this describes GPSD, NUT, Hercules, Roundup, BfW).  

The Emacs history is significantly longer and a bit cruftier than
these, and I've never dealt with a layer of Bazaar before.  Thes
differences do complicate things a bit (I don't normally have to write
custom scripts) but not unmanageably so.

> (Yes, time since conversion is important because the problems are
> usually subtle and don't stick out until much later.)  Detailed
> description of the planned steps during the conversion and how you
> intend to control the quality of each step, will also be appreciated.

I'm enclosing a current copy of the lift script.  I'll add more steps
as I verify them.

As for how I intend to QA them - my strategy has two prongs.  One is
automating everything I can so that I have conditional guarantees of
the form "if tool X is correct, then my results are correct".

The other: historically, I've usually worked in collaboration with a
Mr. Inside, a senior project dev, who checked my work in progress from
a position of intimate knowledge of the project history.

Congratulations, I think you've elected yourself for that job. The
reposurgeon manual is here:

http://www.catb.org/~esr/reposurgeon/reposurgeon.html

> This is great, but doesn't really address the worrisome aspects of the
> conversion we care about.  We no longer care about the elpa branch in
> the bzr repository.  We do care about the few other branches, such as
> emacs-24.  And it is not even clear what will become of those after
> the conversion; the reposurgeon man page cites a limitation related to
> that, allegedly stemming from some (imaginary) bzr confusion between
> branches and repositories, but ends up saying nothing about the
> branches after the conversion.  Will they end up in a single git
> repository, like any other git branches, or won't they?  Will the
> merges between those branches show up as expected in git DAG?  How
> will merges from external branches (such as Org or MH-E) or from local
> feature branches be represented?  Those are much more important issues
> than the ability to split elpa.

You get to tell me what you want to have happen, Mr. Inside.  If
reposurgeon isn't powerful enough to do it, I'll up-gun it until it
is.

Preliminary answer: the git repo after conversion day will, globally
speaking, have the same DAG that the git mirror did before.  Changes will
be localized and consist of (a) commit-clique squashes, and (b) a
few junk branch deletions.

Bazaar's very real branch/repo confusion is probably not relevant,
because my conversion procedure never deals with the Bazaar repository
directly. I start from Andreas's git mirror, which is (presumably)
replicating the branch structure of the entire Bazaar repo every 15
minutes.

If that isn't true, we have some additional problems to solve that
have nothing to do with my tools.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

[-- Attachment #2: emacs.lift --]
[-- Type: text/plain, Size: 34241 bytes --]

# Lift specification for the Emacs conversion
#
# Remove junk lost+found branches.  These contain only dummy commits,
# apparently written as tests.
branch refs/tags/lost+found/12d398afa4d2b77d606b28fe43a2e5200c863419 delete
branch refs/tags/lost+found/5cf718038f760d6dc2e21dee3ce98061a975488a delete
branch refs/tags/lost+found/9a1501510c2a842e8597d695fec05563161a84b7 delete
branch refs/tags/lost+found/d4cd8d7c34cc79dc4abd0fec33cec74afa33e2d0 delete
branch refs/tags/lost+found/e387e14faeea08877c99ddf798d5a4c9f7999c72 delete
branch refs/tags/lost+found/b2e45f822e55882dda1bb80ca8b845d062d6d523 delete
branch refs/tags/lost+found/c003b5d7e29f2b7c3c340d4aa851a61cfa481ec8 delete
#
# Fix up some CVS usernames that apparently didn't get converted during
# the CVS lift. 
#
authors read <<EOF
thomas = Thomas Bushnell <thomas@gnu.ai.mit.edu>
monnier = Stefan Monnier <monnier@iro.umontreal.ca>
EOF
#
# Tag cleanup.  
# All the Emacs tags are lightweight (resets), hence the macro. 
define tagdelete tag tags/{0} delete
define tagrename tag tags/{0} rename tags/{1}
do tagdelete 0.1
do tagdelete Boehm-GC-base
do tagdelete CEDET_BASE
do tagrename EMACS_19_34 emacs-19.34
do tagrename EMACS_20_1 emacs-20.1
do tagrename EMACS_20_2 emacs-20.2
do tagrename EMACS_20_3 emacs-20.3
do tagrename EMACS_20_4 emacs-20.4
do tagrename EMACS_21_1 emacs-21.1
do tagdelete EMACS_21_1_BASE
do tagrename EMACS_21_2 emacs-21.2
do tagrename EMACS_21_3 emacs-21.3
do tagrename EMACS_22_1 emacs-22.1
do tagrename EMACS_22_2 emacs-22.2
do tagrename EMACS_22_3 emacs-22.3
do tagdelete EMACS_22_BRANCHPOINT
do tagrename EMACS_23_1 emacs-23.1
do tagdelete EMACS_23_1_BASE
do tagrename EMACS_23_2 emacs-23.2
do tagrename EMACS_23_3 emacs-23.3
do tagrename EMACS_23_4 emacs-23.4
do tagrename EMACS_PRETEST_21_0_100 emacs-pretest-21.0.100
do tagrename EMACS_PRETEST_21_0_101 emacs-pretest-21.0.101
do tagrename EMACS_PRETEST_21_0_102 emacs-pretest-21.0.102
do tagrename EMACS_PRETEST_21_0_103 emacs-pretest-21.0.103
do tagrename EMACS_PRETEST_21_0_104 emacs-pretest-21.0.104
do tagrename EMACS_PRETEST_21_0_105 emacs-pretest-21.0.105
do tagrename EMACS_PRETEST_21_0_106 emacs-pretest-21.0.106
do tagrename EMACS_PRETEST_21_0_90 emacs-pretest-21.0.90
do tagrename EMACS_PRETEST_21_0_91 emacs-pretest-21.0.91
do tagrename EMACS_PRETEST_21_0_92 emacs-pretest-21.0.92
do tagrename EMACS_PRETEST_21_0_93 emacs-pretest-21.0.93
do tagrename EMACS_PRETEST_21_0_95 emacs-pretest-21.0.95
do tagrename EMACS_PRETEST_21_0_96 emacs-pretest-21.0.96
do tagrename EMACS_PRETEST_21_0_97 emacs-pretest-21.0.97
do tagrename EMACS_PRETEST_21_0_98 emacs-pretest-21.0.98
do tagrename EMACS_PRETEST_21_0_99 emacs-pretest-21.0.99
do tagrename EMACS_PRETEST_21_2_91 emacs-pretest-21.2.91
do tagrename EMACS_PRETEST_21_2_92 emacs-pretest-21.2.92
do tagrename EMACS_PRETEST_21_2_93 emacs-pretest-21.2.93
do tagrename EMACS_PRETEST_21_2_94 emacs-pretest-21.2.94
do tagrename EMACS_PRETEST_21_2_95 emacs-pretest-21.2.95
do tagrename EMACS_PRETEST_22_0_90 emacs-pretest-22.0.90
do tagrename EMACS_PRETEST_22_0_91 emacs-pretest-22.0.91
do tagrename EMACS_PRETEST_22_0_92 emacs-pretest-22.0.92
do tagrename EMACS_PRETEST_22_0_93 emacs-pretest-22.0.93
do tagrename EMACS_PRETEST_22_0_94 emacs-pretest-22.0.94
do tagrename EMACS_PRETEST_22_0_95 emacs-pretest-22.0.95
do tagrename EMACS_PRETEST_22_0_96 emacs-pretest-22.0.96
do tagrename EMACS_PRETEST_22_0_97 emacs-pretest-22.0.97
do tagrename EMACS_PRETEST_22_0_98 emacs-pretest-22.0.98
do tagrename EMACS_PRETEST_22_0_99 emacs-pretest-22.0.99
do tagrename EMACS_PRETEST_22_0_990 emacs-pretest-22.0.990
do tagrename EMACS_PRETEST_22_1_90 emacs-pretest-22.1.90
do tagrename EMACS_PRETEST_22_1_91 emacs-pretest-22.1.91
do tagrename EMACS_PRETEST_22_1_92 emacs-pretest-22.1.92
do tagrename EMACS_PRETEST_22_2_90 emacs-pretest-22.2.90
do tagrename EMACS_PRETEST_22_2_91 emacs-pretest-22.2.91
do tagrename EMACS_PRETEST_22_2_92 emacs-pretest-22.2.92
do tagrename EMACS_PRETEST_23_0_90 emacs-pretest-23.0.90
do tagrename EMACS_PRETEST_23_0_91 emacs-pretest-23.0.91
do tagrename EMACS_PRETEST_23_0_92 emacs-pretest-23.0.92
do tagrename EMACS_PRETEST_23_0_93 emacs-pretest-23.0.93
do tagrename EMACS_PRETEST_23_0_94 emacs-pretest-23.0.94
do tagrename EMACS_PRETEST_23_0_95 emacs-pretest-23.0.95
do tagrename EMACS_PRETEST_23_0_96 emacs-pretest-23.0.96
do tagrename EMACS_PRETEST_23_1_90 emacs-pretest-23.1.90
do tagrename EMACS_PRETEST_23_1_91 emacs-pretest-23.1.91
do tagrename EMACS_PRETEST_23_1_92 emacs-pretest-23.1.92
do tagrename EMACS_PRETEST_23_1_93 emacs-pretest-23.1.93
do tagrename EMACS_PRETEST_23_1_94 emacs-pretest-23.1.94
do tagrename EMACS_PRETEST_23_1_95 emacs-pretest-23.1.95
do tagrename EMACS_PRETEST_23_1_96 emacs-pretest-23.1.96
do tagrename EMACS_PRETEST_23_1_97 emacs-pretest-23.1.97
do tagrename EMACS_PRETEST_23_2_90 emacs-pretest-23.2.90
do tagrename EMACS_PRETEST_23_2_91 emacs-pretest-23.2.91
do tagrename EMACS_PRETEST_23_2_92 emacs-pretest-23.2.92
do tagrename EMACS_PRETEST_23_2_93 emacs-pretest-23.2.93
do tagrename EMACS_PRETEST_23_2_93_1 emacs-pretest-23.2.93.1
do tagrename EMACS_PRETEST_23_2_94 emacs-pretest-23.2.94
do tagrename EMACS_PRETEST_23_3_90 emacs-pretest-23.3.90
do tagrename EMACS_PRETEST_24_0_05 emacs-pretest-24.0.05
do tagrename EMACS_PRETEST_24_0_90 emacs-pretest-24.0.90
do tagrename EMACS_PRETEST_24_0_91 emacs-pretest-24.0.91
do tagrename EMACS_PRETEST_24_0_92 emacs-pretest-24.0.92
do tagrename EMACS_PRETEST_24_0_93 emacs-pretest-24.0.93
do tagrename EMACS_PRETEST_24_0_94 emacs-pretest-24.0.94
do tagrename EMACS_PRETEST_24_0_95 emacs-pretest-24.0.95
do tagdelete Ilya_4_23
do tagdelete Ilya_4_32
do tagdelete Ilya_4_35
do tagdelete Ilya_5_0
do tagdelete Ilya_5_10
do tagdelete Ilya_5_22
do tagdelete Ilya_5_23
do tagdelete Ilya_5_3
do tagdelete Ilya_5_7
do tagdelete NewVC-fileset-BASE
do tagdelete RMAIL-MBOX-BASE
do tagdelete Release_5_25
do tagdelete URL_2004-04-04_01h16
do tagdelete VENDOR-3_3
do tagdelete VENDOR-3_4
do tagdelete VENDOR-3_5
do tagdelete VENDOR-3_6
do tagdelete XFT_JHD_BRANCH_base
do tagdelete after-merge-gnus-5_10
do tagdelete amigados-merge
do tagdelete before-merge-emacs-app-to-trunk
do tagdelete before-merge-gnus-5_10
do tagdelete before-merge-multi-tty-to-trunk
do tagdelete before-merge-unicode-to-trunk
do tagdelete before-miles-orphaned-changes
do tagdelete before-remove-carbon
do tagdelete before-remove-vms
do tagdelete before-thomas-posix1996
do tagdelete branchpoint-5_8
do tagdelete custom_themes_branchpoint
# keep emacs-19.34
# keep emacs-20.1
# keep emacs-20.2
# keep emacs-20.3
# keep emacs-20.4
# keep emacs-21.1
# keep emacs-21.2
# keep emacs-21.3
# keep emacs-22.1
# keep emacs-22.2
# keep emacs-22.3
# keep emacs-23.1
# keep emacs-23.2
# keep emacs-23.3
# keep emacs-23.4
do tagdelete emacs-23/emacs-23.2
# keep emacs-24.0.96
# keep emacs-24.0.97
# keep emacs-24.1
# keep emacs-24.2
# keep emacs-24.2.90
# keep emacs-24.2.91
# keep emacs-24.2.92
# keep emacs-24.2.93
# keep emacs-24.3
do tagdelete emacs-24.3-base
# keep emacs-24.3-rc1
do tagdelete emacs-bidi-base
do tagdelete emacs-unicode-2-base
do tagdelete emacs-unicode-2-pre-sync
do tagdelete emacs-unicode-base
do tagdelete font-backend-base
do tagdelete gcc-2_8_1-980401
do tagdelete gcc-2_8_1-980402
do tagdelete gcc-2_8_1-980407
do tagdelete gcc-2_8_1-980412
do tagdelete gcc-2_8_1-980413
do tagdelete gcc-2_8_1-980419
do tagdelete gcc-2_8_1-980426
do tagdelete gcc-2_8_1-980502
do tagdelete gcc-2_8_1-980513
do tagdelete gcc-2_8_1-980525
do tagdelete gcc-2_8_1-980529
do tagdelete gcc-2_8_1-980608
do tagdelete gcc-2_8_1-980609
do tagdelete gcc-2_8_1-980627
do tagdelete gcc-2_8_1-980705
do tagdelete gcc-2_8_1-980718
do tagdelete gcc-2_8_1-980811
do tagdelete gcc-2_8_1-980813
do tagdelete gcc-2_8_1-980928
do tagdelete gcc-2_8_1-980929
do tagdelete gcc-2_8_1-RELEASE
do tagdelete gcc_2_8_1-980315
do tagdelete gcc_2_8_1-980929
do tagdelete glibc-2_0_2
do tagdelete glibc-2_0_4
do tagdelete gnumach-release-1-1
do tagdelete gnumach-release-1-1-1
do tagdelete gnumach-release-1-1-2
do tagdelete gnumach-release-1-1-3
do tagdelete gnus-5_10-branchpoint
do tagdelete gnus-5_10-post-merge-josefsson
do tagdelete gnus-5_10-post-merge-yamaoka
do tagdelete gnus-5_10-pre-merge-josefsson
do tagdelete gnus-5_10-pre-merge-yamaoka
do tagdelete handa-temp-tag
do tagdelete hurd-release-0-2
do tagdelete jimb-sync-Nov-3-1992
do tagdelete kfs_20030524_post
do tagdelete kfs_20030524_pre
do tagdelete lexbind-base
do tagdelete libc-1-90
do tagdelete libc-1-91
do tagdelete libc-1-92
do tagdelete libc-1-93
do tagdelete libc-950402
do tagdelete libc-950411
do tagdelete libc-950722
do tagdelete libc-950723
do tagdelete libc-950922
do tagdelete libc-951016
do tagdelete libc-951018
do tagdelete libc-951029
do tagdelete libc-951031
do tagdelete libc-951101
do tagdelete libc-951102
do tagdelete libc-951103
do tagdelete libc-951104
do tagdelete libc-951105
do tagdelete libc-951106
do tagdelete libc-951107
do tagdelete libc-951108
do tagdelete libc-951109
do tagdelete libc-951110
do tagdelete libc-951111
do tagdelete libc-951112
do tagdelete libc-951113
do tagdelete libc-951114
do tagdelete libc-951115
do tagdelete libc-951116
do tagdelete libc-951117
do tagdelete libc-951118
do tagdelete libc-951119
do tagdelete libc-951120
do tagdelete libc-951121
do tagdelete libc-951122
do tagdelete libc-951123
do tagdelete libc-951124
do tagdelete libc-951125
do tagdelete libc-951126
do tagdelete libc-951127
do tagdelete libc-951128
do tagdelete libc-951129
do tagdelete libc-951130
do tagdelete libc-951201
do tagdelete libc-951202
do tagdelete libc-951203
do tagdelete libc-951204
do tagdelete libc-951206
do tagdelete libc-951208
do tagdelete libc-951209
do tagdelete libc-951210
do tagdelete libc-951211
do tagdelete libc-951212
do tagdelete libc-951213
do tagdelete libc-951214
do tagdelete libc-951215
do tagdelete libc-951216
do tagdelete libc-951217
do tagdelete libc-951218
do tagdelete libc-951219
do tagdelete libc-951220
do tagdelete libc-951221
do tagdelete libc-951222
do tagdelete libc-951223
do tagdelete libc-951224
do tagdelete libc-951225
do tagdelete libc-951226
do tagdelete libc-951227
do tagdelete libc-951228
do tagdelete libc-951229
do tagdelete libc-951230
do tagdelete libc-951231
do tagdelete libc-960101
do tagdelete libc-960102
do tagdelete libc-960103
do tagdelete libc-960104
do tagdelete libc-960105
do tagdelete libc-960106
do tagdelete libc-960107
do tagdelete libc-960108
do tagdelete libc-960109
do tagdelete libc-960110
do tagdelete libc-960111
do tagdelete libc-960112
do tagdelete libc-960113
do tagdelete libc-960114
do tagdelete libc-960115
do tagdelete libc-960116
do tagdelete libc-960117
do tagdelete libc-960118
do tagdelete libc-960119
do tagdelete libc-960120
do tagdelete libc-960121
do tagdelete libc-960122
do tagdelete libc-960123
do tagdelete libc-960124
do tagdelete libc-960125
do tagdelete libc-960126
do tagdelete libc-960127
do tagdelete libc-960128
do tagdelete libc-960129
do tagdelete libc-960130
do tagdelete libc-960131
do tagdelete libc-960201
do tagdelete libc-960202
do tagdelete libc-960203
do tagdelete libc-960204
do tagdelete libc-960205
do tagdelete libc-960206
do tagdelete libc-960207
do tagdelete libc-960208
do tagdelete libc-960209
do tagdelete libc-960210
do tagdelete libc-960211
do tagdelete libc-960212
do tagdelete libc-960213
do tagdelete libc-960214
do tagdelete libc-960215
do tagdelete libc-960216
do tagdelete libc-960217
do tagdelete libc-960218
do tagdelete libc-960219
do tagdelete libc-960220
do tagdelete libc-960221
do tagdelete libc-960222
do tagdelete libc-960223
do tagdelete libc-960224
do tagdelete libc-960225
do tagdelete libc-960226
do tagdelete libc-960227
do tagdelete libc-960228
do tagdelete libc-960229
do tagdelete libc-960302
do tagdelete libc-960303
do tagdelete libc-960304
do tagdelete libc-960305
do tagdelete libc-960306
do tagdelete libc-960307
do tagdelete libc-960308
do tagdelete libc-960309
do tagdelete libc-960310
do tagdelete libc-960311
do tagdelete libc-960312
do tagdelete libc-960313
do tagdelete libc-960314
do tagdelete libc-960315
do tagdelete libc-960316
do tagdelete libc-960317
do tagdelete libc-960318
do tagdelete libc-960319
do tagdelete libc-960320
do tagdelete libc-960321
do tagdelete libc-960322
do tagdelete libc-960323
do tagdelete libc-960324
do tagdelete libc-960325
do tagdelete libc-960326
do tagdelete libc-960327
do tagdelete libc-960328
do tagdelete libc-960329
do tagdelete libc-960330
do tagdelete libc-960331
do tagdelete libc-960401
do tagdelete libc-960402
do tagdelete libc-960403
do tagdelete libc-960404
do tagdelete libc-960405
do tagdelete libc-960406
do tagdelete libc-960407
do tagdelete libc-960408
do tagdelete libc-960409
do tagdelete libc-960410
do tagdelete libc-960411
do tagdelete libc-960412
do tagdelete libc-960413
do tagdelete libc-960414
do tagdelete libc-960415
do tagdelete libc-960416
do tagdelete libc-960417
do tagdelete libc-960418
do tagdelete libc-960419
do tagdelete libc-960420
do tagdelete libc-960421
do tagdelete libc-960422
do tagdelete libc-960423
do tagdelete libc-960424
do tagdelete libc-960425
do tagdelete libc-960426
do tagdelete libc-960427
do tagdelete libc-960428
do tagdelete libc-960429
do tagdelete libc-960430
do tagdelete libc-960501
do tagdelete libc-960502
do tagdelete libc-960503
do tagdelete libc-960504
do tagdelete libc-960505
do tagdelete libc-960506
do tagdelete libc-960507
do tagdelete libc-960508
do tagdelete libc-960509
do tagdelete libc-960510
do tagdelete libc-960511
do tagdelete libc-960512
do tagdelete libc-960513
do tagdelete libc-960514
do tagdelete libc-960515
do tagdelete libc-960516
do tagdelete libc-960517
do tagdelete libc-960518
do tagdelete libc-960519
do tagdelete libc-960520
do tagdelete libc-960521
do tagdelete libc-960522
do tagdelete libc-960523
do tagdelete libc-960524
do tagdelete libc-960525
do tagdelete libc-960526
do tagdelete libc-960527
do tagdelete libc-960528
do tagdelete libc-960529
do tagdelete libc-960530
do tagdelete libc-960531
do tagdelete libc-960601
do tagdelete libc-960602
do tagdelete libc-960603
do tagdelete libc-960604
do tagdelete libc-960605
do tagdelete libc-960606
do tagdelete libc-960607
do tagdelete libc-960608
do tagdelete libc-960609
do tagdelete libc-960610
do tagdelete libc-960611
do tagdelete libc-960612
do tagdelete libc-960613
do tagdelete libc-960614
do tagdelete libc-960615
do tagdelete libc-960616
do tagdelete libc-960617
do tagdelete libc-960618
do tagdelete libc-960619
do tagdelete libc-960620
do tagdelete libc-960621
do tagdelete libc-960622
do tagdelete libc-960623
do tagdelete libc-960624
do tagdelete libc-960625
do tagdelete libc-960626
do tagdelete libc-960627
do tagdelete libc-960628
do tagdelete libc-960629
do tagdelete libc-960630
do tagdelete libc-960701
do tagdelete libc-960702
do tagdelete libc-960703
do tagdelete libc-960704
do tagdelete libc-960705
do tagdelete libc-960706
do tagdelete libc-960707
do tagdelete libc-960708
do tagdelete libc-960709
do tagdelete libc-960710
do tagdelete libc-960711
do tagdelete libc-960712
do tagdelete libc-960713
do tagdelete libc-960714
do tagdelete libc-960715
do tagdelete libc-960716
do tagdelete libc-960717
do tagdelete libc-960718
do tagdelete libc-960719
do tagdelete libc-960720
do tagdelete libc-960721
do tagdelete libc-960722
do tagdelete libc-960723
do tagdelete libc-960724
do tagdelete libc-960725
do tagdelete libc-960726
do tagdelete libc-960727
do tagdelete libc-960728
do tagdelete libc-960729
do tagdelete libc-960730
do tagdelete libc-960731
do tagdelete libc-960801
do tagdelete libc-960802
do tagdelete libc-960803
do tagdelete libc-960804
do tagdelete libc-960805
do tagdelete libc-960806
do tagdelete libc-960807
do tagdelete libc-960808
do tagdelete libc-960809
do tagdelete libc-960810
do tagdelete libc-960811
do tagdelete libc-960812
do tagdelete libc-960813
do tagdelete libc-960814
do tagdelete libc-960815
do tagdelete libc-960816
do tagdelete libc-960817
do tagdelete libc-960818
do tagdelete libc-960819
do tagdelete libc-960820
do tagdelete libc-960821
do tagdelete libc-960822
do tagdelete libc-960823
do tagdelete libc-960824
do tagdelete libc-960825
do tagdelete libc-960826
do tagdelete libc-960827
do tagdelete libc-960828
do tagdelete libc-960829
do tagdelete libc-960830
do tagdelete libc-960831
do tagdelete libc-960901
do tagdelete libc-960902
do tagdelete libc-960903
do tagdelete libc-960904
do tagdelete libc-960905
do tagdelete libc-960906
do tagdelete libc-960907
do tagdelete libc-960908
do tagdelete libc-960909
do tagdelete libc-960910
do tagdelete libc-960911
do tagdelete libc-960912
do tagdelete libc-960913
do tagdelete libc-960918
do tagdelete libc-960919
do tagdelete libc-960920
do tagdelete libc-960921
do tagdelete libc-960922
do tagdelete libc-960923
do tagdelete libc-960925
do tagdelete libc-960926
do tagdelete libc-960927
do tagdelete libc-960928
do tagdelete libc-960929
do tagdelete libc-961001
do tagdelete libc-961004
do tagdelete libc-961005
do tagdelete libc-961006
do tagdelete libc-961007
do tagdelete libc-961008
do tagdelete libc-961009
do tagdelete libc-961010
do tagdelete libc-961011
do tagdelete libc-961012
do tagdelete libc-961013
do tagdelete libc-961014
do tagdelete libc-961015
do tagdelete libc-961016
do tagdelete libc-961017
do tagdelete libc-961018
do tagdelete libc-961019
do tagdelete libc-961020
do tagdelete libc-961021
do tagdelete libc-961022
do tagdelete libc-961023
do tagdelete libc-961024
do tagdelete libc-961025
do tagdelete libc-961026
do tagdelete libc-961027
do tagdelete libc-961028
do tagdelete libc-961029
do tagdelete libc-961030
do tagdelete libc-961031
do tagdelete libc-961101
do tagdelete libc-961102
do tagdelete libc-961103
do tagdelete libc-961104
do tagdelete libc-961105
do tagdelete libc-961106
do tagdelete libc-961107
do tagdelete libc-961108
do tagdelete libc-961109
do tagdelete libc-961110
do tagdelete libc-961111
do tagdelete libc-961114
do tagdelete libc-961115
do tagdelete libc-961116
do tagdelete libc-961117
do tagdelete libc-961118
do tagdelete libc-961119
do tagdelete libc-961120
do tagdelete libc-961121
do tagdelete libc-961203
do tagdelete libc-961204
do tagdelete libc-961205
do tagdelete libc-961206
do tagdelete libc-961207
do tagdelete libc-961208
do tagdelete libc-961209
do tagdelete libc-961210
do tagdelete libc-961211
do tagdelete libc-961212
do tagdelete libc-961213
do tagdelete libc-961214
do tagdelete libc-961215
do tagdelete libc-961216
do tagdelete libc-961217
do tagdelete libc-961218
do tagdelete libc-961219
do tagdelete libc-961220
do tagdelete libc-961221
do tagdelete libc-961222
do tagdelete libc-961223
do tagdelete libc-961224
do tagdelete libc-961225
do tagdelete libc-961226
do tagdelete libc-961227
do tagdelete libc-961228
do tagdelete libc-961229
do tagdelete libc-961230
do tagdelete libc-961231
do tagdelete libc-970101
do tagdelete libc-970102
do tagdelete libc-970103
do tagdelete libc-970104
do tagdelete libc-970105
do tagdelete libc-970106
do tagdelete libc-970107
do tagdelete libc-970108
do tagdelete libc-970109
do tagdelete libc-970110
do tagdelete libc-970111
do tagdelete libc-970112
do tagdelete libc-970113
do tagdelete libc-970114
do tagdelete libc-970115
do tagdelete libc-970116
do tagdelete libc-970117
do tagdelete libc-970118
do tagdelete libc-970119
do tagdelete libc-970120
do tagdelete libc-970121
do tagdelete libc-970122
do tagdelete libc-970123
do tagdelete libc-970124
do tagdelete libc-970125
do tagdelete libc-970126
do tagdelete libc-970127
do tagdelete libc-970128
do tagdelete libc-970129
do tagdelete libc-970130
do tagdelete libc-970131
do tagdelete libc-970201
do tagdelete libc-970202
do tagdelete libc-970203
do tagdelete libc-970204
do tagdelete libc-970205
do tagdelete libc-970206
do tagdelete libc-970207
do tagdelete libc-970208
do tagdelete libc-970209
do tagdelete libc-970210
do tagdelete libc-970211
do tagdelete libc-970212
do tagdelete libc-970213
do tagdelete libc-970214
do tagdelete libc-970215
do tagdelete libc-970216
do tagdelete libc-970217
do tagdelete libc-970218
do tagdelete libc-970219
do tagdelete libc-970220
do tagdelete libc-970221
do tagdelete libc-970222
do tagdelete libc-970223
do tagdelete libc-970224
do tagdelete libc-970225
do tagdelete libc-970226
do tagdelete libc-970227
do tagdelete libc-970228
do tagdelete libc-970301
do tagdelete libc-970302
do tagdelete libc-970303
do tagdelete libc-970304
do tagdelete libc-970305
do tagdelete libc-970306
do tagdelete libc-970307
do tagdelete libc-970308
do tagdelete libc-970309
do tagdelete libc-970310
do tagdelete libc-970311
do tagdelete libc-970312
do tagdelete libc-970313
do tagdelete libc-970314
do tagdelete libc-970315
do tagdelete libc-970316
do tagdelete libc-970317
do tagdelete libc-970318
do tagdelete libc-970319
do tagdelete libc-970320
do tagdelete libc-970321
do tagdelete libc-970322
do tagdelete libc-970323
do tagdelete libc-970324
do tagdelete libc-970325
do tagdelete libc-970326
do tagdelete libc-970327
do tagdelete libc-970328
do tagdelete libc-970329
do tagdelete libc-970330
do tagdelete libc-970331
do tagdelete libc-970401
do tagdelete libc-970402
do tagdelete libc-970403
do tagdelete libc-970404
do tagdelete libc-970405
do tagdelete libc-970406
do tagdelete libc-970407
do tagdelete libc-970408
do tagdelete libc-970409
do tagdelete libc-970410
do tagdelete libc-970411
do tagdelete libc-970412
do tagdelete libc-970413
do tagdelete libc-970414
do tagdelete libc-970415
do tagdelete libc-970416
do tagdelete libc-970417
do tagdelete libc-970418
do tagdelete libc-970419
do tagdelete libc-970420
do tagdelete libc-970421
do tagdelete libc-970422
do tagdelete libc-970423
do tagdelete libc-970424
do tagdelete libc-970425
do tagdelete libc-970426
do tagdelete libc-970427
do tagdelete libc-970428
do tagdelete libc-970429
do tagdelete libc-970430
do tagdelete libc-970501
do tagdelete libc-970502
do tagdelete libc-970503
do tagdelete libc-970504
do tagdelete libc-970505
do tagdelete libc-970506
do tagdelete libc-970507
do tagdelete libc-970508
do tagdelete libc-970509
do tagdelete libc-970510
do tagdelete libc-970511
do tagdelete libc-970512
do tagdelete libc-970513
do tagdelete libc-970514
do tagdelete libc-970515
do tagdelete libc-970516
do tagdelete libc-970517
do tagdelete libc-970518
do tagdelete libc-970519
do tagdelete libc-970520
do tagdelete libc-970521
do tagdelete libc-970522
do tagdelete libc-970523
do tagdelete libc-970524
do tagdelete libc-970525
do tagdelete libc-970526
do tagdelete libc-970527
do tagdelete libc-970528
do tagdelete libc-970529
do tagdelete libc-970530
do tagdelete libc-970531
do tagdelete libc-970601
do tagdelete libc-970602
do tagdelete libc-970603
do tagdelete libc-970604
do tagdelete libc-970605
do tagdelete libc-970606
do tagdelete libc-970607
do tagdelete libc-970608
do tagdelete libc-970609
do tagdelete libc-970610
do tagdelete libc-970611
do tagdelete libc-970612
do tagdelete libc-970613
do tagdelete libc-970614
do tagdelete libc-970615
do tagdelete libc-970616
do tagdelete libc-970617
do tagdelete libc-970618
do tagdelete libc-970619
do tagdelete libc-970620
do tagdelete libc-970621
do tagdelete libc-970622
do tagdelete libc-970624
do tagdelete libc-970625
do tagdelete libc-970626
do tagdelete libc-970627
do tagdelete libc-970628
do tagdelete libc-970629
do tagdelete libc-970630
do tagdelete libc-970701
do tagdelete libc-970702
do tagdelete libc-970703
do tagdelete libc-970704
do tagdelete libc-970705
do tagdelete libc-970707
do tagdelete libc-970708
do tagdelete libc-970709
do tagdelete libc-970710
do tagdelete libc-970713
do tagdelete libc-970715
do tagdelete libc-970717
do tagdelete libc-970718
do tagdelete libc-970719
do tagdelete libc-970720
do tagdelete libc-970721
do tagdelete libc-970722
do tagdelete libc-970723
do tagdelete libc-970724
do tagdelete libc-970725
do tagdelete libc-970726
do tagdelete libc-970727
do tagdelete libc-970728
do tagdelete libc-970729
do tagdelete libc-970730
do tagdelete libc-970731
do tagdelete libc-970801
do tagdelete libc-970802
do tagdelete libc-970803
do tagdelete libc-970804
do tagdelete libc-970805
do tagdelete libc-970806
do tagdelete libc-970807
do tagdelete libc-970808
do tagdelete libc-970809
do tagdelete libc-970810
do tagdelete libc-970811
do tagdelete libc-970812
do tagdelete libc-970813
do tagdelete libc-970814
do tagdelete libc-970815
do tagdelete libc-970816
do tagdelete libc-970817
do tagdelete libc-970818
do tagdelete libc-970819
do tagdelete libc-970820
do tagdelete libc-970821
do tagdelete libc-970822
do tagdelete libc-970823
do tagdelete libc-970824
do tagdelete libc-970825
do tagdelete libc-970826
do tagdelete libc-970827
do tagdelete libc-970828
do tagdelete libc-970829
do tagdelete libc-970830
do tagdelete libc-970831
do tagdelete libc-970901
do tagdelete libc-970902
do tagdelete libc-970903
do tagdelete libc-970904
do tagdelete libc-970905
do tagdelete libc-970906
do tagdelete libc-970907
do tagdelete libc-970908
do tagdelete libc-970911
do tagdelete libc-970912
do tagdelete libc-970913
do tagdelete libc-970914
do tagdelete libc-970915
do tagdelete libc-970916
do tagdelete libc-970917
do tagdelete libc-970918
do tagdelete libc-970919
do tagdelete libc-970920
do tagdelete libc-970921
do tagdelete libc-970922
do tagdelete libc-970923
do tagdelete libc-970924
do tagdelete libc-970925
do tagdelete libc-970926
do tagdelete libc-970927
do tagdelete libc-970928
do tagdelete libc-970929
do tagdelete libc-970930
do tagdelete libc-971001
do tagdelete libc-971018
do tagdelete libc-971019
do tagdelete libc-971020
do tagdelete libc-971021
do tagdelete libc-971022
do tagdelete libc-971023
do tagdelete libc-971024
do tagdelete libc-971025
do tagdelete libc-971026
do tagdelete libc-971027
do tagdelete libc-971028
do tagdelete libc-971029
do tagdelete libc-971030
do tagdelete libc-971031
do tagdelete libc-971101
do tagdelete libc-971102
do tagdelete libc-971103
do tagdelete libc-971104
do tagdelete libc-971105
do tagdelete libc-971106
do tagdelete libc-971107
do tagdelete libc-971108
do tagdelete libc-971109
do tagdelete libc-971110
do tagdelete libc-971111
do tagdelete libc-971112
do tagdelete libc-971113
do tagdelete libc-971114
do tagdelete libc-971115
do tagdelete libc-971116
do tagdelete libc-971117
do tagdelete libc-971118
do tagdelete libc-971120
do tagdelete libc-971121
do tagdelete libc-971122
do tagdelete libc-971123
do tagdelete libc-971124
do tagdelete libc-971125
do tagdelete libc-971126
do tagdelete libc-971127
do tagdelete libc-971128
do tagdelete libc-971129
do tagdelete libc-971130
do tagdelete libc-971201
do tagdelete libc-971203
do tagdelete libc-971204
do tagdelete libc-971205
do tagdelete libc-971206
do tagdelete libc-971207
do tagdelete libc-971208
do tagdelete libc-971209
do tagdelete libc-971210
do tagdelete libc-971211
do tagdelete libc-971212
do tagdelete libc-971213
do tagdelete libc-971214
do tagdelete libc-971217
do tagdelete libc-971218
do tagdelete libc-971219
do tagdelete libc-971220
do tagdelete libc-971221
do tagdelete libc-971222
do tagdelete libc-971223
do tagdelete libc-971224
do tagdelete libc-971225
do tagdelete libc-971226
do tagdelete libc-971227
do tagdelete libc-971228
do tagdelete libc-971229
do tagdelete libc-971230
do tagdelete libc-971231
do tagdelete libc-980103
do tagdelete libc-980104
do tagdelete libc-980105
do tagdelete libc-980106
do tagdelete libc-980107
do tagdelete libc-980108
do tagdelete libc-980109
do tagdelete libc-980110
do tagdelete libc-980111
do tagdelete libc-980112
do tagdelete libc-980114
do tagdelete libc-980115
do tagdelete libc-980116
do tagdelete libc-980117
do tagdelete libc-980118
do tagdelete libc-980119
do tagdelete libc-980120
do tagdelete libc-980121
do tagdelete libc-980122
do tagdelete libc-980123
do tagdelete libc-980124
do tagdelete libc-980125
do tagdelete libc-980126
do tagdelete libc-980127
do tagdelete libc-980128
do tagdelete libc-980129
do tagdelete libc-980130
do tagdelete libc-980212
do tagdelete libc-980213
do tagdelete libc-980214
do tagdelete libc-980215
do tagdelete libc-980216
do tagdelete libc-980217
do tagdelete libc-980218
do tagdelete libc-980219
do tagdelete libc-980220
do tagdelete libc-980221
do tagdelete libc-980222
do tagdelete libc-980223
do tagdelete libc-980224
do tagdelete libc-980225
do tagdelete libc-980226
do tagdelete libc-980227
do tagdelete libc-980228
do tagdelete libc-980301
do tagdelete libc-980302
do tagdelete libc-980303
do tagdelete libc-980304
do tagdelete libc-980306
do tagdelete libc-980307
do tagdelete libc-980308
do tagdelete libc-980309
do tagdelete libc-980310
do tagdelete libc-980311
do tagdelete libc-980312
do tagdelete libc-980313
do tagdelete libc-980314
do tagdelete libc-980315
do tagdelete libc-980316
do tagdelete libc-980317
do tagdelete libc-980318
do tagdelete libc-980319
do tagdelete libc-980320
do tagdelete libc-980321
do tagdelete libc-980322
do tagdelete libc-980323
do tagdelete libc-980324
do tagdelete libc-980325
do tagdelete libc-980326
do tagdelete libc-980327
do tagdelete libc-980328
do tagdelete libc-980329
do tagdelete libc-980330
do tagdelete libc-980331
do tagdelete libc-980401
do tagdelete libc-980402
do tagdelete libc-980403
do tagdelete libc-980404
do tagdelete libc-980405
do tagdelete libc-980406
do tagdelete libc-980407
do tagdelete libc-980408
do tagdelete libc-980409
do tagdelete libc-980410
do tagdelete libc-980411
do tagdelete libc-980412
do tagdelete libc-980413
do tagdelete libc-980414
do tagdelete libc-980428
do tagdelete libc-980429
do tagdelete libc-980430
do tagdelete libc-980501
do tagdelete libc-980502
do tagdelete libc-980503
do tagdelete libc-980504
do tagdelete libc-980505
do tagdelete libc-980506
do tagdelete libc-980507
do tagdelete libc-980508
do tagdelete libc-980509
do tagdelete libc-980510
do tagdelete libc-980512
do tagdelete libc-980513
do tagdelete libc-980514
do tagdelete libc-980515
do tagdelete libc-980516
do tagdelete libc-980517
do tagdelete libc-980518
do tagdelete libc-980519
do tagdelete libc-980520
do tagdelete libc-980521
do tagdelete libc-980522
do tagdelete libc-980523
do tagdelete libc-980524
do tagdelete libc-980525
do tagdelete libc-980526
do tagdelete libc-980527
do tagdelete libc-980528
do tagdelete libc-980529
do tagdelete libc-980530
do tagdelete libc-980531
do tagdelete libc-980601
do tagdelete libc-980602
do tagdelete libc-980603
do tagdelete libc-980604
do tagdelete libc-980605
do tagdelete libc-980606
do tagdelete libc-980607
do tagdelete libc-980608
do tagdelete libc-980609
do tagdelete libc-980610
do tagdelete libc-980611
do tagdelete libc-980612
do tagdelete libc-980613
do tagdelete libc-980614
do tagdelete libc-980615
do tagdelete libc-980616
do tagdelete libc-980617
do tagdelete libc-980618
do tagdelete libc-980619
do tagdelete libc-980620
do tagdelete libc-980621
do tagdelete libc-980622
do tagdelete libc-980623
do tagdelete libc-980624
do tagdelete libc-980625
do tagdelete libc-980626
do tagdelete libc-980627
do tagdelete libc-980628
do tagdelete libc-980629
do tagdelete libc-980630
do tagdelete libc-980701
do tagdelete libc-980702
do tagdelete libc-980703
do tagdelete libc-980704
do tagdelete libc-980705
do tagdelete libc-980706
do tagdelete libc-980707
do tagdelete libc-980708
do tagdelete libc-980709
do tagdelete libc-980710
do tagdelete libc-980711
do tagdelete libc-980712
do tagdelete libc-980713
do tagdelete libc-980714
do tagdelete libc-980715
do tagdelete libc-980716
do tagdelete libc-980717
do tagdelete libc-980718
do tagdelete libc-980719
do tagdelete libc-980720
do tagdelete libc20x-970306
do tagdelete libc20x-97031
do tagdelete libc20x-970316
do tagdelete libc20x-970318
do tagdelete libc20x-970319
do tagdelete libc20x-970404
do tagdelete libc20x-970417
do tagdelete libc_1_09
do tagdelete lisp-bob
do tagdelete make-3-72-10
do tagdelete make-3-72-11
do tagdelete make-3-72-12
do tagdelete make-3-72-13
do tagdelete make-3-72-9
do tagdelete make-3-73
do tagdelete make-3-73-1
do tagdelete make-3-73-2
do tagdelete make-3-73-3
do tagdelete make-3-74
do tagdelete make-3-74-1
do tagdelete make-3-74-2
do tagdelete make-3-74-3
do tagdelete make-3-74-4
do tagdelete make-3-74-5
do tagdelete make-3-74-6
do tagdelete make-3-74-7
do tagdelete make-3-75
do tagdelete make-3-75-1
do tagdelete make-3-75-91
do tagdelete make-3-75-92
do tagdelete make-3-75-93
do tagdelete make-3-76
do tagdelete make-3-76-1
do tagdelete make-3-77
do tagdelete merge-multi-tty-to-trunk
do tagdelete merge-unicode-to-trunk
do tagdelete mh-e-7_85
do tagdelete mh-e-7_90
do tagdelete mh-e-7_91
do tagdelete mh-e-7_92
do tagdelete mh-e-7_93
do tagdelete mh-e-7_94
do tagdelete mh-e-7_95
do tagdelete mh-e-8.2.90
do tagdelete mh-e-8.2.91
do tagdelete mh-e-8.2.92
do tagdelete mh-e-8.2.93
do tagdelete mh-e-8.3
do tagdelete mh-e-8.3.1
do tagdelete mh-e-8.4
do tagdelete mh-e-8.5
do tagdelete mh-e-8_0
do tagdelete mh-e-8_0_1
do tagdelete mh-e-8_0_2
do tagdelete mh-e-8_0_3
do tagdelete mh-e-8_1
do tagdelete mh-e-8_2
do tagdelete mh-e-doc-7_94
do tagdelete mh-e-doc-8.3
do tagdelete mh-e-doc-8.4
do tagdelete mh-e-doc-8.5
do tagdelete mh-e-doc-8_0
do tagdelete mh-e-doc-8_0_1
do tagdelete mh-e-doc-8_0_3
do tagdelete mh-e-doc-8_1
do tagdelete mh-e-doc-8_2
do tagdelete multi-tty-base
do tagdelete patches_21_0_base
do tagdelete raeburn-tag-4-for-export
do tagdelete raeburn-tag-5-for-export
do tagdelete raeburn-tag-6-for-export
do tagdelete raeburn-tag-7-for-export
do tagdelete release-0-0
do tagdelete release-0-1
do tagdelete release-1-0
do tagdelete remove-carbon
do tagdelete remove-vms
do tagdelete root-libc-2_0_x-branch
do tagdelete small-dump-base
do tagdelete sml-mode-6.0
do tagdelete sml-mode-6.1
do tagdelete tmp_pcl_tag_131034C
# keep ttn-vms-21-2-B2
# keep ttn-vms-21-2-B3
# keep ttn-vms-21-2-B4
do tagdelete unicode-post-font-backend
do tagdelete unicode-pre-font-backend
do tagdelete unicode-xft-base
do tagdelete v0.1
do tagdelete v0.1.0
do tagdelete v0.1.1
do tagdelete v0.1.2
do tagdelete v0.1.3
do tagdelete v0.2.0
do tagdelete v0.3.0
do tagdelete v1_7i
do tagdelete v2007-Sep-10
do tagdelete xwidget-0.2
do tagdelete zsh-merge-ognus-1
do tagdelete zsh-sync-ognus-2
do tagdelete zsh-sync-ognus-3
undefine tagdelete
undefine tagrename
#
# TO DO: drop the elpa branch.

Re: Goals for repo conversion day

[Werner LEMBERG]

> By the time I did groff, late last year, my tools and procedures for
> normal cases were pretty well routinized and bulletproofed.  You can
> read about them here:

Speaking as the (almost former) groff maintainer: The conversion
wasn't perfect at the first time – it suffered the *empty ChangeLog*
problem due to a too small time window for merging related multiple
CVS commits into a single git commit –, but Eric posted the git
repository for downloading (or rather, he posted a fast-import dump
with instructions how to proceed), and after some improvements I was
satisfied with the result, which is now the official git repository.
Thanks again for that, Eric!

> The other: historically, I've usually worked in collaboration with a
> Mr. Inside, a senior project dev, who checked my work in progress
> from a position of intimate knowledge of the project history.
> 
> Congratulations, I think you've elected yourself for that job.

Hehe :-)


    Werner

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Mon, 27 Jan 2014 06:16:53 +0100 (CET)
> Cc: eliz@gnu.org, schwab@linux-m68k.org, emacs-devel@gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> Speaking as the (almost former) groff maintainer: The conversion
> wasn't perfect at the first time -- it suffered the *empty ChangeLog*
> problem due to a too small time window for merging related multiple
> CVS commits into a single git commit --, but Eric posted the git
> repository for downloading (or rather, he posted a fast-import dump
> with instructions how to proceed), and after some improvements I was
> satisfied with the result, which is now the official git repository.

With all due respect, Werner, I don't think your experience with Groff
is enough to reassure us that the problems are imaginary.  With its
strictly linear history (not a single branch or merge!), and not even
2300 revisions, Groff's history is almost 2 orders of magnitude
smaller than Emacs's.  I also don't see any traces of converted
references to old revision numbers (apologies if I missed them).  And
you only switched a month and a half ago, so cannot yet claim a
long-term experience.

Re: Goals for repo conversion day

[Werner LEMBERG]

>> Speaking as the (almost former) groff maintainer: The conversion
>> wasn't perfect at the first time -- it suffered the *empty
>> ChangeLog* problem due to a too small time window for merging
>> related multiple CVS commits into a single git commit --, but Eric
>> posted the git repository for downloading (or rather, he posted a
>> fast-import dump with instructions how to proceed), and after some
>> improvements I was satisfied with the result, which is now the
>> official git repository.
> 
> With all due respect, Werner, I don't think your experience with
> Groff is enough to reassure us that the problems are imaginary.

Oh, you got me wrong.  My main statement is rather that the git
repository can be easily inspected before it becomes `official'.


    Werner

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Mon, 27 Jan 2014 18:42:56 +0100 (CET)
> Cc: esr@thyrsus.com, emacs-devel@gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> 
> > With all due respect, Werner, I don't think your experience with
> > Groff is enough to reassure us that the problems are imaginary.
> 
> Oh, you got me wrong.  My main statement is rather that the git
> repository can be easily inspected before it becomes `official'.

Sorry about my misunderstanding.

Re: Goals for repo conversion day

[Andreas Schwab]

There is one more thing in the history that may be worth fixing, though
it may lead to quite a bit of manual work.  Before emacs switched to CVS
the sources were kept in RCS, and files were deleted by renaming the RCS
file prefixing its name with "=" (to keep the history, since RCS doesn't
have the concept of an attic like CVS).  The obstacle is that there is
no record of this deletion except in the ChangeLog file, and at that
time the ChangeLog files weren't kept in RCS (they were versioned by
numbered backups only).  So in order to find the exact point in time
when the file has been deleted for real one would have to grep the
ChangeLog file for mentioning the deletion and look up the surrounding
text in the commit log.  For example, the file lisp/speedbspec.el only
exists as lisp/=speedbspec.el in the current history, added in commit
73bf48f.  The next mentioning of the string "speedbspec.el" was in
commit ab18f00, which corresponds to the point where the file was
deleted.  The corresponding changelog file (lisp/ChangeLog.7) has this
entry:

1998-07-10  Eric M. Ludlam  <zappo@ultranet.com>

        * speedbspec.el: Deleted; now integrated into speedbar.el.
        * speedbar.el: More commentary.

Note that the first line of the entry isn't mentioned in commit ab18f00,
but we now know that the file was deleted here.  The next commit
touching this file is 1c3e7fb with the message "properly mark Attic
files as deleted", where the file was finally deleted from the tree.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Goals for repo conversion day

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> There is one more thing in the history that may be worth fixing, though
> it may lead to quite a bit of manual work.  Before emacs switched to CVS
> the sources were kept in RCS, and files were deleted by renaming the RCS
> file prefixing its name with "=" (to keep the history, since RCS doesn't
> have the concept of an attic like CVS).  The obstacle is that there is
> no record of this deletion except in the ChangeLog file, and at that
> time the ChangeLog files weren't kept in RCS (they were versioned by
> numbered backups only).  So in order to find the exact point in time
> when the file has been deleted for real one would have to grep the
> ChangeLog file for mentioning the deletion and look up the surrounding
> text in the commit log.  For example, the file lisp/speedbspec.el only
> exists as lisp/=speedbspec.el in the current history, added in commit
> 73bf48f.  The next mentioning of the string "speedbspec.el" was in
> commit ab18f00, which corresponds to the point where the file was
> deleted.  The corresponding changelog file (lisp/ChangeLog.7) has this
> entry:
> 
> 1998-07-10  Eric M. Ludlam  <zappo@ultranet.com>
> 
>         * speedbspec.el: Deleted; now integrated into speedbar.el.
>         * speedbar.el: More commentary.
> 
> Note that the first line of the entry isn't mentioned in commit ab18f00,
> but we now know that the file was deleted here.  The next commit
> touching this file is 1c3e7fb with the message "properly mark Attic
> files as deleted", where the file was finally deleted from the tree.
> 
> Andreas.

Yuck.

Fortunately, I have a search primitive that will find all these instances.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Goals for repo conversion day

[Ulrich Mueller]

>>>>> On Mon, 27 Jan 2014, Eric S. Raymond wrote:

> Andreas Schwab <schwab@linux-m68k.org>:
>> There is one more thing in the history that may be worth fixing,
>> though it may lead to quite a bit of manual work. Before emacs
>> switched to CVS the sources were kept in RCS, and files were
>> deleted by renaming the RCS file prefixing its name with "=" (to
>> keep the history, since RCS doesn't have the concept of an attic
>> like CVS). The obstacle is that there is no record of this deletion
>> except in the ChangeLog file, and at that time the ChangeLog files
>> weren't kept in RCS (they were versioned by numbered backups only).
>> So in order to find the exact point in time when the file has been
>> deleted for real one would have to grep the ChangeLog file for
>> mentioning the deletion and look up the surrounding text in the
>> commit log. For example, the file lisp/speedbspec.el only exists as
>> lisp/=speedbspec.el in the current history, added in commit
>> 73bf48f. The next mentioning of the string "speedbspec.el" was in
>> commit ab18f00, which corresponds to the point where the file was
>> deleted. The corresponding changelog file (lisp/ChangeLog.7) has
>> this entry:
>>
>> 1998-07-10  Eric M. Ludlam  <zappo@ultranet.com>
>> 
>>         * speedbspec.el: Deleted; now integrated into speedbar.el.
>>         * speedbar.el: More commentary.
>>
>> Note that the first line of the entry isn't mentioned in commit
>> ab18f00, but we now know that the file was deleted here. The next
>> commit touching this file is 1c3e7fb with the message "properly
>> mark Attic files as deleted", where the file was finally deleted
>> from the tree.

> Yuck.

> Fortunately, I have a search primitive that will find all these
> instances.

So, this will take care of deleted files. However, I wonder how
renaming of files was handled in pre-CVS times? My guess would be that
the RCS file would have been renamed. How could one find out if and
when this happened?

Ulrich

Re: Goals for repo conversion day

[Andreas Schwab]

Ulrich Mueller <ulm@gentoo.org> writes:

> So, this will take care of deleted files. However, I wonder how
> renaming of files was handled in pre-CVS times? My guess would be that
> the RCS file would have been renamed.

Good point.

> How could one find out if and when this happened?

Only from the ChangeLog.  Especially the files in src/m were frequently
renamed.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Goals for repo conversion day

[David Kastrup]

Andreas Schwab <schwab@linux-m68k.org> writes:

> Ulrich Mueller <ulm@gentoo.org> writes:
>
>> So, this will take care of deleted files. However, I wonder how
>> renaming of files was handled in pre-CVS times? My guess would be that
>> the RCS file would have been renamed.
>
> Good point.
>
>> How could one find out if and when this happened?
>
> Only from the ChangeLog.  Especially the files in src/m were frequently
> renamed.

This is getting silly.  When "renaming" files in either RCS _or_ CVS,
_either_ the file history _or_ the directory state (meaning that an
older checkout did no longer compile because files appeared under their
new name) was _lost_.  Why should we _now_ try rewriting distant history
that was good enough when it was still young?

-- 
David Kastrup

What to do about the attic files

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> > How could one find out if and when this happened?
> 
> Only from the ChangeLog.  Especially the files in src/m were frequently
> renamed.

I do not hold out much hope that we can reconstruct those renames
completely.  But I will look into it.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Goals for repo conversion day

[Eli Zaretskii]

> Date: Sun, 26 Jan 2014 19:33:12 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org

I initially wrote a much longer and detailed response, but eventually
decided to scrap it.  I doubt that this discussion is going anywhere,
since you are determined to do what you think is right, no matter what
I say or show.

So this is a much shorter version, which only touches on a few issues
that I don't yet understand, or where you didn't yet say enough.

> To illustrate my methods, I fixed this by adding those revnos to the 
> ChangeLog section of the map file I enclosed in my last mail (it is
> the file FOSSILS in my conversion directory).  Then I ran a Python
> script called 'decorate.py' that patched in the corresponding action
> stamps.  The point is that I didn't have to do the lookup by hand; the
> fixup took less time to do than to describe.

What will the "patched" ChangeLog entries and log messages look like?
I think it's important that we see that in advance.

> The Bazaar portion of the history isn't the problem, the CVS part is.
> There are many instances in the CVS part of the Emacs history that
> look something like this:
> 
> 1. Eli changes file A and commits it
> 2. Eli changes file B and commits it with an identical change comment.

That's where you are wrong: in the CVS part of Emacs history, the
change comments for these two commits will usually be different.

Here's a typical example:

  ------------------------------------------------------------
  revno: 40047
  committer: Eli Zaretskii <eliz@gnu.org>
  timestamp: Fri 2001-10-19 16:32:49 +0000
  message:
    *** empty log message ***
  modified:
    lisp/ChangeLog
  ------------------------------------------------------------
  revno: 40046
  committer: Eli Zaretskii <eliz@gnu.org>
  timestamp: Fri 2001-10-19 16:30:11 +0000
  message:
    (auto-mode-alist): Associate .indent.pro with
    Fundamental mode.  Suggested by Samuel Padgett
    <spadgett1@nc.rr.com>.
  modified:
    lisp/files.el

Here's another, more complex one (several commits in different
directories, followed by commits of the 2 ChangeLog files describing
those changes):

  ------------------------------------------------------------
  revno: 45027
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:57:09 +0000
  message:
    *** empty log message ***
  modified:
    lisp/ChangeLog
    src/ChangeLog
  ------------------------------------------------------------
  revno: 45026
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:56:19 +0000
  message:
    (display-time-mail-directory, display-time-mail-function): Doc fixes.
  modified:
    lisp/time.el
  ------------------------------------------------------------
  revno: 45025
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:55:02 +0000
  message:
    (Special Properties): Add index entry and xref for tooltips.
  modified:
    lispref/text.texi
  ------------------------------------------------------------
  revno: 45024
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:53:39 +0000
  message:
    Comment change.
  modified:
    src/eval.c
    src/fns.c
  ------------------------------------------------------------
  revno: 45023
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:48:09 +0000
  message:
    Delete ediff-hook.el page.
  modified:
    lisp/loaddefs.el
  ------------------------------------------------------------
  revno: 45022
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:46:40 +0000
  message:
    (Frequire): Error if called while preparing to dump.
  modified:
    src/fns.c
  ------------------------------------------------------------
  revno: 45021
  committer: Richard M. Stallman <rms@gnu.org>
  timestamp: Tue 2002-04-30 17:46:26 +0000
  message:
    (do_autoload): Error if called while preparing to dump.
  modified:
    src/eval.c

There probably are multiple commits with the same commit message, but
they are a tiny minority.

> There is another case common in the Emacs history that can be
> coalesced.  That is: a file modification immediately followed by a
> ChangeLog change describing it

What do you mean by "immediately followed"?  Is there some time window
involved, or some other condition?  If so, please spell them out.

And please take a good look at the examples I pointed to in my
previous message and above: those are very good examples of what most
of our CVS history looks like.  Instead of talking about hypothetical
problems, let's talk about real ones, the ones relevant to _this_
project.

> The other: historically, I've usually worked in collaboration with a
> Mr. Inside, a senior project dev, who checked my work in progress from
> a position of intimate knowledge of the project history.
>
> Congratulations, I think you've elected yourself for that job.

Thanks, but manual QA can only go that far.  I was talking about an
automated one.  There's no way a human, even a genius such as myself,
could inspect any significant portion of a 120k commit history.  There
has to be automated verification of some kind, otherwise the quality
of the result is anyone's guess, your record of converting other
projects notwithstanding.

Besides, I'm very much in doubt that cooperating with me will be a
good idea: we see things too differently, and based on this
discussion, will be unable to convince each other.

Bzr's "confusion" between branches and repositories

[Eli Zaretskii]

> Date: Sun, 26 Jan 2014 19:33:12 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: schwab@linux-m68k.org, emacs-devel@gnu.org
> 
> Bazaar's very real branch/repo confusion is probably not relevant,

Since you are saying this in quite a few of your blogs and articles,
let's finish this once and for all: there is no confusion.  Bazaar
works on branches, period.  Bazaar is deeply branch-centric, right
down to the level of its DAG topology and the way it handles merges.
A bzr repository is just a convenience for storing several related
branches in a way that saves disk storage, that's all.  With the
exception of a handful of commands that make some sense in a repo
(such as "bzr branches" and "bzr info"), _all_ of the bzr commands
work and are meaningful only in a branch.

If you want another evidence that bzr repositories are just containers
with little user-level impact, hear this: last versions of bzr support
co-located branches, but the result, i.e. several co-located branches
that live in a single disk directory, is _not_ a repository, and is
not referenced as such.  They didn't even agree on a name, before the
development went dark.

You found the "confusion" in what bzr-fast-import does, but that is an
import tool, and has nothing to do with day-to-day bzr ops.  It does
what it does because you will generally import from a repository of
another VCS.

So please, stop disseminating this fallacy.

Re: Bzr's "confusion" between branches and repositories

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Since you are saying this in quite a few of your blogs and articles,
> let's finish this once and for all: there is no confusion.  Bazaar
> works on branches, period.  Bazaar is deeply branch-centric, right
> down to the level of its DAG topology and the way it handles merges.
> A bzr repository is just a convenience for storing several related
> branches in a way that saves disk storage, that's all.

This is not true for most repositories.  The branches are usually
connected together by having a common root.  This is also the case of
the Emacs repository: *all* branches (except the old elpa branch) have a
single root in common.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Bzr's "confusion" between branches and repositories

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: esr@thyrsus.com,  emacs-devel@gnu.org
> Date: Mon, 27 Jan 2014 17:47:24 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > A bzr repository is just a convenience for storing several related
> > branches in a way that saves disk storage, that's all.
> 
> This is not true for most repositories.  The branches are usually
> connected together by having a common root.

Yes, that's what I meant by "related".  Sorry for being inaccurate.

Re: Bzr's "confusion" between branches and repositories

[Eli Zaretskii]

> Date: Mon, 27 Jan 2014 18:53:37 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: esr@thyrsus.com, emacs-devel@gnu.org
> 
> > From: Andreas Schwab <schwab@linux-m68k.org>
> > Cc: esr@thyrsus.com,  emacs-devel@gnu.org
> > Date: Mon, 27 Jan 2014 17:47:24 +0100
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > A bzr repository is just a convenience for storing several related
> > > branches in a way that saves disk storage, that's all.
> > 
> > This is not true for most repositories.  The branches are usually
> > connected together by having a common root.
> 
> Yes, that's what I meant by "related".  Sorry for being inaccurate.

Btw, it's possible to have branches without a common root (or any
common revisions) in a bzr repo, it just doesn't make sense, because
there will be no disk space savings.

Re: Goals for repo conversion day

[Glenn Morris]

Eli Zaretskii wrote:

> Comments and opinions by others are welcome.

I agree wholeheartedly with basically everything you said.

Re: Goals for repo conversion day

[Andreas Schwab]

My goal was to preserve as much history as possible, not to rewrite it.
The only editing I introduced were artificial merge commits, since they
could be added without altering the actual file contents.

(A lot of damage was done to the glibc repository when it was
converted.  I wanted to avoid that.)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Goals for repo conversion day

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

() "Eric S. Raymond" <esr@thyrsus.com>
() Sat, 25 Jan 2014 09:06:38 -0500

   (a) Seamless history browsing.

This is false fruit.  Archeologists LIKE the seams.  Smoothing the repo
(possible) but not the external references into the repo (impossible) is
at best a half-assed "solution", at worst a ridiculous and never-ending
font of WOMBAT laments.  I think the best we can hope for is a decent
map (and of course nice Emacs-based tools for grokking it in full :-D).

   (b) Information preservation for any future move to another VCS.

   (c) Cryptosigning so that future release integrity is protected and
   historical release reconstructions can be trusted (that is, assuming
   we don't believe the code history has already been corrupted).

   (d) Avoidance of metadata representations that are easily stripped or
   scrambled if someone gets momentarily forgetful in the future.

If these can be achieved w/o undue repo futzing, that would be best.  We
will have a repo that is (permanently) ugly up to the point where (going
forward) we begin to maintain it more beautifully, but that's just fine.
In the long run, ugly truth is better than pretty lies.

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Goals for repo conversion day

[Eric S. Raymond]

[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]

Thien-Thi Nguyen <ttn@gnu.org>:
>    (a) Seamless history browsing.
> 
> This is false fruit.  Archeologists LIKE the seams.

That is true.  But beware of misleading analogies!  Archeologists like
the seams mainly because stratigraphy helps them date things.  This is a problem
historians looking through a version-control repository won't have.

>                                                     Smoothing the repo
> (possible) but not the external references into the repo (impossible) is
> at best a half-assed "solution", at worst a ridiculous and never-ending
> font of WOMBAT laments.  I think the best we can hope for is a decent
> map (and of course nice Emacs-based tools for grokking it in full :-D).

We will have a decent map.  Whether anyone cares enough to build those
tools is another question.

> If these can be achieved w/o undue repo futzing, that would be best.

I intend to publish the conversion script here for review well before
it is applied; you will be able to judge for yourself whether it is
"undue", and suggest improvements.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Goals for repo conversion day

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

() "Eric S. Raymond" <esr@thyrsus.com>
() Sat, 25 Jan 2014 14:54:01 -0500

   That is true.  But beware of misleading analogies!  Archeologists
   like the seams mainly because stratigraphy helps them date things.

Repo contents is first order information.  Higher order is: What are
the structures that hold the repo contents (i.e., what is the VCS)?;
What are the factors in its choice?; Why are hackers so human?; ...

   This is a problem historians looking through a version-control
   repository won't have.

Right.  They will have this problem from the extra-repo references in
mailing list archives (public and private), bug tracker databases,
blogs, etc.  If you really want to help historians, the maps (and tools)
will be what they will thank you for the most, for the insight afforded.
(Insert Alan Perlis quote here.)

Historians might thank you, anyway, for an "antiseptic" repo, but
privately refer to an unsullied copy, for the Real Deal, a place where
Free and Hippocratic programs are composed and run (composably :-D).

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Goals for repo conversion day

[Eric S. Raymond]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

Thien-Thi Nguyen <ttn@gnu.org>:
> () "Eric S. Raymond" <esr@thyrsus.com>
> () Sat, 25 Jan 2014 14:54:01 -0500
> 
>    That is true.  But beware of misleading analogies!  Archeologists
>    like the seams mainly because stratigraphy helps them date things.
> 
> Repo contents is first order information.  Higher order is: What are
> the structures that hold the repo contents (i.e., what is the VCS)?;
> What are the factors in its choice?; Why are hackers so human?; ...

I agree that may be of intyerest at some far future time.  For that reason,
and others, I do recommend theat old repositories be archived rather than
deleted out of hand.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: The git mirror is *very* badly screwed up

[Stefan Monnier]

> Andreas says I should do nothing with that branch.  But if elpa has
> its own repo now, probably it should be trimmed out on conversion day.

You can drop that branch in the new Git.


        Stefan

Re: The git mirror is *very* badly screwed up

[Eric S. Raymond]

Stefan Monnier <monnier@iro.umontreal.ca>:
> > Andreas says I should do nothing with that branch.  But if elpa has
> > its own repo now, probably it should be trimmed out on conversion day.
> 
> You can drop that branch in the new Git.

Acknowledged.  I've added that to-do to a comment in the lift file.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>