Progress on the git repo cleanup

Progress on the git repo cleanup

[Eric S. Raymond]

Some time ago I posted a list of desiderata for a cleaned-up gir
repository.  I now have a lift script that achieves all of these.
Here is the current list

# 1. Map some remnant CVS user names to DVCS IDs
#
# 2. Turn fossil Bazaar and CVS references into action stamps.  
#
# 3. Identify and convert .cvsignore and .bzrignore files.
#
# 4. Rename and deletion fixups for RCS attic files
#
# 5. Squash commit cliques that were not properly merged during the Bazaar lift.
#
# 6. Incorporate fixes-bug properties from Bazaar.
#
# 7. Massage bug URLs in change comments into a canonical form.

One minor issue remains; the *only* 4-digit Bazaar reference (to 3176)
does not lift for some reason.  I can fix that.

The conversion machinery is at 

git@gitorious.org:emacs-transition/emacs-transition.git

so others can audit my procedure.  Everything is documented.  In the
near future I will push a copy of the cleaned-up repo to gitorious.

What this means is that we can now pull the switch anytime Stefan says.

The only practical problem is that a full run of the lift script takes
around ten hours. You can reduce the number of those I have to do by
not embedding any more Bazaar commit numbers in ChangeLogs or commit
comments.

I've been sent a list of alternative forms of committer names that
could be normalized, but I recall there was some resistance to doing
this.  Stefan, do you have a policy preference?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

The politician attempts to remedy the evil by increasing the very thing
that caused the evil in the first place: legal plunder.
	-- Frederick Bastiat

Re: Progress on the git repo cleanup

[David Reitter]

Eric,

What is the procedure for those of us that keep a separate branch around, which has multiple (50+) merges from the mainline or Emacs 23, and many private changes?

Thanks,
David


On Mar 6, 2014, at 1:21 PM, Eric S. Raymond <esr@thyrsus.com> wrote:

> Some time ago I posted a list of desiderata for a cleaned-up gir
> repository.  I now have a lift script that achieves all of these.
> Here is the current list
> 
> # 1. Map some remnant CVS user names to DVCS IDs
> #
> # 2. Turn fossil Bazaar and CVS references into action stamps.  
> #
> # 3. Identify and convert .cvsignore and .bzrignore files.
> #
> # 4. Rename and deletion fixups for RCS attic files
> #
> # 5. Squash commit cliques that were not properly merged during the Bazaar lift.
> #
> # 6. Incorporate fixes-bug properties from Bazaar.
> #
> # 7. Massage bug URLs in change comments into a canonical form.
> 
> One minor issue remains; the *only* 4-digit Bazaar reference (to 3176)
> does not lift for some reason.  I can fix that.
> 
> The conversion machinery is at 
> 
> git@gitorious.org:emacs-transition/emacs-transition.git
> 
> so others can audit my procedure.  Everything is documented.  In the
> near future I will push a copy of the cleaned-up repo to gitorious.
> 
> What this means is that we can now pull the switch anytime Stefan says.
> 
> The only practical problem is that a full run of the lift script takes
> around ten hours. You can reduce the number of those I have to do by
> not embedding any more Bazaar commit numbers in ChangeLogs or commit
> comments.
> 
> I've been sent a list of alternative forms of committer names that
> could be normalized, but I recall there was some resistance to doing
> this.  Stefan, do you have a policy preference?
> -- 
> 		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
> 
> The politician attempts to remedy the evil by increasing the very thing
> that caused the evil in the first place: legal plunder.
> 	-- Frederick Bastiat
> 

Re: Progress on the git repo cleanup

[Andreas Schwab]

David Reitter <david.reitter@gmail.com> writes:

> What is the procedure for those of us that keep a separate branch around, which has multiple (50+) merges from the mainline or Emacs 23, and many private changes?

The minimum requirement is a mapping of every referenced commit in
git.sv.gnu.org/emacs to a commit in the converted repo (it doesn't have
to be one-to-one, but can be many-to-one).  This can then be used to
create a script based on bzr-fastimport to convert private branches.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Progress on the git repo cleanup

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On Mar 6, 2014, at 3:15 PM, Andreas Schwab <schwab@linux-m68k.org> wrote:

> David Reitter <david.reitter@gmail.com> writes:
> 
>> What is the procedure for those of us that keep a separate branch around, which has multiple (50+) merges from the mainline or Emacs 23, and many private changes?
> 
> The minimum requirement is a mapping of every referenced commit in
> git.sv.gnu.org/emacs to a commit in the converted repo (it doesn't have
> to be one-to-one, but can be many-to-one).  This can then be used to
> create a script based on bzr-fastimport to convert private branches.

Right.  In my case, I’d like to see the refs from the official Git mirror included (as it exists now).  My branches are based on that.  I think I’m not the only one.

- David

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4151 bytes --]

Re: Progress on the git repo cleanup

[Eric S. Raymond]

David Reitter <david.reitter@gmail.com>:
> What is the procedure for those of us that keep a separate branch
> around, which has multiple (50+) merges from the mainline or Emacs
> 23, and many private changes?

The first question you need to settle is whether you need to keep the
change comments for your branch.  If not, the procedure is very simple:

1. New repo is put up.

2. You pull it.

3. You start a new branch.

4. You rm -fr and copy in your version.

5. You commit and push it, creating a new branch in the master repo.

If you need to keep the change comments, Andreas has a more complicated 
procedure in mind which I will help him set up.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Progress on the git repo cleanup

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 1129 bytes --]

On Mar 6, 2014, at 3:49 PM, Eric S. Raymond <esr@thyrsus.com> wrote:

> David Reitter <david.reitter@gmail.com>:
>> What is the procedure for those of us that keep a separate branch
>> around, which has multiple (50+) merges from the mainline or Emacs
>> 23, and many private changes?
> 
> The first question you need to settle is whether you need to keep the
> change comments for your branch.  

I started Aquamacs 9 years ago, and I have about 5 years worth of history in the branch.  I have been careful to be a good citizen in committing reasonably logical and documented changes that can be merged into the main branch.
No, I don’t want to lose my history.

> If not, the procedure is very simple:

If it weren’t for your dedicated and hard work while communicating with everyone here, I would find it somewhat insulting to be given this trivial “trash the history” solution.

> If you need to keep the change comments, Andreas has a more complicated 
> procedure in mind which I will help him set up.

I’m very much looking forward to this.  
Can I provide you with a reasonably complex test case?


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4151 bytes --]

Re: Progress on the git repo cleanup

[Eric S. Raymond]

David Reitter <david.reitter@gmail.com>:
> If it weren’t for your dedicated and hard work while communicating
> with everyone here, I would find it somewhat insulting to be given
> this trivial “trash the history” solution.

I intended no offense.  I don't know what people don't know.

> > If you need to keep the change comments, Andreas has a more complicated 
> > procedure in mind which I will help him set up.
> 
> I’m very much looking forward to this.  
> Can I provide you with a reasonably complex test case?

Please do.  I don't know exactly what Andreas plans to do, but he has told
me what he needs to do it (that is, a map of existing hashes to new ones).
and I will try to generate that.

Another alternative, possibly simpler, is that you could push the branch to
the public Bazaar repository and let me know you have done so.  Then it
would get converted automatically the next time I do a lift.

Is there any reason not to do things that way?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Progress on the git repo cleanup

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]

On Mar 6, 2014, at 5:00 PM, Eric S. Raymond <esr@thyrsus.com> wrote:
> 
> Another alternative, possibly simpler, is that you could push the branch to
> the public Bazaar repository and let me know you have done so.  Then it
> would get converted automatically the next time I do a lift.
> 
> Is there any reason not to do things that way?

My branches are git branches.

- David


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4151 bytes --]

Re: Progress on the git repo cleanup

[Andreas Schwab]

David Reitter <david.reitter@gmail.com> writes:

> On Mar 6, 2014, at 5:00 PM, Eric S. Raymond <esr@thyrsus.com> wrote:
>> 
>> Another alternative, possibly simpler, is that you could push the branch to
>> the public Bazaar repository and let me know you have done so.  Then it
>> would get converted automatically the next time I do a lift.
>> 
>> Is there any reason not to do things that way?
>
> My branches are git branches.

In that case doing a git filter-branch rewriting the parents of all
non-public commit should do it, based on the old/new mapping.

(I was assuming private bzr branches when wrote the other mail, although
I should have known better.)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Progress on the git repo cleanup

[Eric S. Raymond]

David Reitter <david.reitter@gmail.com>:
> My branches are git branches.

I was missing that crucial bit.  OK, now I think I understand what Andreas
intends to do  I'll work on my end of it.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Progress on the git repo cleanup

[Andreas Schwab]

I have another request for cleanup: it has been pointed out that my repo
has four commits that are rejected by modern git because the author does
not conform to the usual conventions:

commit 676f341
Author:  <Thomas>
Date:   Wed May 7 19:19:04 1997 +0000

commit 7fb42a2
Author:  <Thomas>
Date:   Thu Nov 21 22:20:09 1996 +0000

commit b563928
Author:  <Thomas>
Date:   Thu Nov 21 21:51:00 1996 +0000

commit dd17baf
Author:  <Thomas>
Date:   Thu Nov 21 21:43:48 1996 +0000

The author should be changed to "Thomas Bushnell, BSG <thomas@gnu.org>".

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Progress on the git repo cleanup

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> I have another request for cleanup: it has been pointed out that my repo
> has four commits that are rejected by modern git because the author does
> not conform to the usual conventions:
> 
> commit 676f341
> Author:  <Thomas>
> Date:   Wed May 7 19:19:04 1997 +0000
> 
> commit 7fb42a2
> Author:  <Thomas>
> Date:   Thu Nov 21 22:20:09 1996 +0000
> 
> commit b563928
> Author:  <Thomas>
> Date:   Thu Nov 21 21:51:00 1996 +0000
> 
> commit dd17baf
> Author:  <Thomas>
> Date:   Thu Nov 21 21:43:48 1996 +0000
> 
> The author should be changed to "Thomas Bushnell, BSG <thomas@gnu.org>".

It is already done (well, excpt for the "BSG" - what does that mean?).
The relevant section of the lift file looks like this:

#
# Fix up some CVS usernames that apparently didn't get converted during
# the CVS lift. 
#
authors read <<EOF
thomas = Thomas Bushnell <thomas@gnu.ai.mit.edu>
monnier = Stefan Monnier <monnier@iro.umontreal.ca>
EOF

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Progress on the git repo cleanup

[Karl Fogel]

"Eric S. Raymond" <esr@thyrsus.com> writes:
>It is already done (well, excpt for the "BSG" - what does that mean?).

Brotherhood of Saint Gregory.  Thomas Bushnell is a Gregorian friar; his
name was "Michael Bushnell" (the same 'mib' who worked at the FSF on the
GNU Hurd for some years) before this.  I believe the s/Michael/Thomas/
change was due to his joining the BSG.

It is not a monastic retreat from the world or anything like that, by
the way: Thomas works at Google now.

I guess Git doesn't have a metadata slot for recording religious orders
with the author name :-).

-Karl

Re: Progress on the git repo cleanup

[Eric S. Raymond]

Karl Fogel <kfogel@red-bean.com>:
> I guess Git doesn't have a metadata slot for recording religious orders
> with the author name :-).

Um, no. :-)

First time I've seen any git attribution requested or proposed with
an order membership or title of nobility or anything like that.  Makes
me wonder if I should add "KSC" to mine.

KSC = "Keeper of the Sacred Chao"
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Progress on the git repo cleanup

[Karl Fogel]

"Eric S. Raymond" <esr@thyrsus.com> writes:
>Karl Fogel <kfogel@red-bean.com>:
>> I guess Git doesn't have a metadata slot for recording religious orders
>> with the author name :-).
>
>Um, no. :-)
>
>First time I've seen any git attribution requested or proposed with
>an order membership or title of nobility or anything like that.  Makes
>me wonder if I should add "KSC" to mine.
>
>KSC = "Keeper of the Sacred Chao"

I'm going for "Viscount" once we're on the new system.

Re: Progress on the git repo cleanup

[David Kastrup]

Karl Fogel <kfogel@red-bean.com> writes:

> "Eric S. Raymond" <esr@thyrsus.com> writes:
>>Karl Fogel <kfogel@red-bean.com>:
>>> I guess Git doesn't have a metadata slot for recording religious orders
>>> with the author name :-).
>>
>>Um, no. :-)
>>
>>First time I've seen any git attribution requested or proposed with
>>an order membership or title of nobility or anything like that.  Makes
>>me wonder if I should add "KSC" to mine.
>>
>>KSC = "Keeper of the Sacred Chao"
>
> I'm going for "Viscount" once we're on the new system.

Sounds like Sesame Street's "The Count" for Visual Basic.

-- 
David Kastrup