Bazaar migration status?

Bazaar migration status?

[joakim]

I have a couple of local Emacs bzr branches(xwidget and the "pinnable
windows" feature). These are kept in sync with Emacs CVS through Jason
Earls experimental emacs bzr mirror.

To make a long story short, I've made a misstake and my branches are
no longer syncable from Jasons repository.

<whining>

Ok, so now I'm grumpy and everything, but I feel this wouldnt have
happened if Emacs would have migrated from CVS to Bazaar now.

Why couldn't I just wait until Emacs had done the migration? Because I
wanted to do some Emacs develoment without feeling my intestines being
eaten by CVS.

</whining>

-- 
Joakim Verona

Re: Bazaar migration status?

[Stefan Monnier]

> To make a long story short, I've made a misstake and my branches are
> no longer syncable from Jasons repository.

Are you sure it was a mistake of yours: Jason's repository has had
problems (around the same time the CVS crashed, tho I think it was just
a coincidence).  I think Jason was trying to get a new one going, but
I don't know its status (and it wouldn't be syncable with the old
one :-( )

But, yes, I'm also eager to see us move away from CVS.


        Stefan

Re: Bazaar migration status?

[joakim]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> To make a long story short, I've made a misstake and my branches are
>> no longer syncable from Jasons repository.
>
> Are you sure it was a mistake of yours: Jason's repository has had
> problems (around the same time the CVS crashed, tho I think it was just
> a coincidence).  I think Jason was trying to get a new one going, but
> I don't know its status (and it wouldn't be syncable with the old
> one :-( )

My mistake was not expecting this to happen with an experimental repos,
and not properly exporting my patches before trying to get up to date
with the new repos. So its my fault, and now I have a lot more work to
do before I can get up to speed again.

My point is I would expect this not to happen with a canonical Emacs bzr
repos, even if it had some flaws in the translation data.


> But, yes, I'm also eager to see us move away from CVS.

Can we translate this mutual eagerness into actual action?

FWIW when I encounter this situation in my professional role I usually
just disable committing to the old repos, leaving it on-line, and start
fresh with the new repos. 

>
>
>         Stefan
-- 
Joakim Verona

Re: Bazaar migration status?

[Stefan Monnier]

>> But, yes, I'm also eager to see us move away from CVS.
> Can we translate this mutual eagerness into actual action?

First thing is to get a sample Bazaar repository installed on
bzr.sv.gnu.org.  Apparently, the Bazaar repository will have to use the
brand-new "brisbane" (can't remember the official name) format, so the
version of bzr on bzr.sv.gnu.org will have to be upgraded, together with
the surrounding tools.


        Stefan

Re: Bazaar migration status?

[Daniel Clemente]

El vie, jul 17 2009 a les 19:56, Stefan Monnier va escriure:
>> Can we translate this mutual eagerness into actual action?
>
> First thing is to get a sample Bazaar repository installed on bzr.sv.gnu.org. 

  How will people be able to track progress on this? Maybe via Emacs' or Savannah's bug tracker?

  There's a similar request for this in Savannah ([1]) but it's outdated and development has stopped.
[1]: http://savannah.gnu.org/support/?106612


-- Daniel

Re: Bazaar migration status?

[Karl Fogel]

Daniel Clemente <dcl441-bugs@yahoo.com> writes:
>   How will people be able to track progress on this? Maybe via Emacs' or Savannah's bug tracker?
>
>   There's a similar request for this in Savannah ([1]) but it's outdated and development has stopped.
> [1]: http://savannah.gnu.org/support/?106612

Not stopped, just stalled.  I'm back on it now, though note that Sylvan
Beucler has been the one actually doing the work on Savannah.

I just filed https://savannah.gnu.org/support/index.php?106951 to
request that Bazaar be upgraded to 1.17 on Savannah, so we'll have
support for the "2a" format (see the notes about Loggerhead there).

I've also updated http://savannah.gnu.org/support/?106612 to point to
#106951.  It wasn't clear how to formally express the dependency
relationship (at the bottom of #106612, the UI implies it's possible, by
saying "Depends on the following items: None found"... but I don't see
where to actually _create_ a dependency), so I just put the reference in
a comment.

In a separate post, I'll hit up Jason Earl and Andreas Schwab yet again
to create a "2a" format Emacs Bazaar repository, and hopefully this will
be The One this time.

-Karl

Re: Bazaar migration status?

[Chong Yidong]

joakim@verona.se writes:

>> But, yes, I'm also eager to see us move away from CVS.
>
> Can we translate this mutual eagerness into actual action?
>
> FWIW when I encounter this situation in my professional role I usually
> just disable committing to the old repos, leaving it on-line, and start
> fresh with the new repos.

We're supposedly waiting for the latest Bazaar version to stabilize
first, see

  http://lists.gnu.org/archive/html/emacs-devel/2009-06/msg00540.html

Re: Bazaar migration status?

[Karl Fogel]

Chong Yidong <cyd@stupidchicken.com> writes:
> joakim@verona.se writes:
>>> But, yes, I'm also eager to see us move away from CVS.
>>
>> Can we translate this mutual eagerness into actual action?
>>
>> FWIW when I encounter this situation in my professional role I usually
>> just disable committing to the old repos, leaving it on-line, and start
>> fresh with the new repos.
>
> We're supposedly waiting for the latest Bazaar version to stabilize
> first, see
>
>   http://lists.gnu.org/archive/html/emacs-devel/2009-06/msg00540.html

Well, we could switch now.  I've just been AWOL on some other stuff
(that unfortunately cannot wait) and have not been able to work on the
Emacs bzr switchover.  My other stuff should be finished in the near
future (near meaning weeks, not months), and then I do intend to come
back to this.  Sorry for the delay, and of course I don't mean to
discourage anyone else from picking up the reins if they want to!

-Karl

Re: Bazaar migration status?

[joakim]

Karl Fogel <karl.fogel@canonical.com> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>> joakim@verona.se writes:
>>>> But, yes, I'm also eager to see us move away from CVS.
>>>
>>> Can we translate this mutual eagerness into actual action?

This comment was meant to be fun but sounds harsh now that I read it.
I meant that I can help if theres something I could do, however, I do
not know what in particular to do to help.

>>>
>>> FWIW when I encounter this situation in my professional role I usually
>>> just disable committing to the old repos, leaving it on-line, and start
>>> fresh with the new repos.
>>
>> We're supposedly waiting for the latest Bazaar version to stabilize
>> first, see
>>
>>   http://lists.gnu.org/archive/html/emacs-devel/2009-06/msg00540.html
>
> Well, we could switch now.  I've just been AWOL on some other stuff
> (that unfortunately cannot wait) and have not been able to work on the
> Emacs bzr switchover.  My other stuff should be finished in the near
> future (near meaning weeks, not months), and then I do intend to come
> back to this.  Sorry for the delay, and of course I don't mean to
> discourage anyone else from picking up the reins if they want to!
>
> -Karl
-- 
Joakim Verona

Re: Bazaar migration status?

[Ken Raeburn]

I don't recall seeing it... was there ever an answer as to whether the  
git mirror on savannah would continue to work after the migration?

Re: Bazaar migration status?

[Stefan Monnier]

> I don't recall seeing it... was there ever an answer as to whether the git
> mirror on savannah would continue to work after the migration?

The git mirror is completely independent, so there's no reason to think
that it will be affected.  But I don't maintain it, so take it with
a grain of salt.


        Stefan

Re: Bazaar migration status?

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

On Jul 20, 2009, at 2:35 PM, Stefan Monnier wrote:

>> I don't recall seeing it... was there ever an answer as to whether  
>> the git
>> mirror on savannah would continue to work after the migration?
>
> The git mirror is completely independent, so there's no reason to  
> think
> that it will be affected.  But I don't maintain it, so take it with
> a grain of salt.

As far as I know, the git mirror is synced to the CVS repository.  If  
the CVS repo goes away or isn't updated any longer, then neither is  
the git mirror.

Will we have a CVS  (read-only) mirror?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 2193 bytes --]

Re: Bazaar migration status?

[Stefan Monnier]

> As far as I know, the git mirror is synced to the CVS repository.
> If the CVS repo goes away or isn't updated any longer, then neither is
> the git mirror.

I see no reason why the people who setup&maintain the Git mirror will
not be interested in having a Git mirror after we switch to any other
VCS (except for Git, obviously).


        Stefan

Re: Bazaar migration status?

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> As far as I know, the git mirror is synced to the CVS repository.
>> If the CVS repo goes away or isn't updated any longer, then neither is
>> the git mirror.
>
> I see no reason why the people who setup&maintain the Git mirror will
> not be interested in having a Git mirror after we switch to any other
> VCS (except for Git, obviously).

Sure, tho I think the cvs-to-git syncing tool is by far the most
developed; I think it was actually David (Reiter) who asked on the git
list about bzr-to-git syncing tools, and IIRC, the replies didn't seem
so encouraging.

[Another problem is that even if another syncing tool is available,
I'm not sure what changing to another source-VCS/tool means for the
existing history in git...]

-Miles

-- 
Virtues, n. pl. Certain abstentions.

Re: Bazaar migration status?

[Stefan Monnier]

> Sure, tho I think the cvs-to-git syncing tool is by far the most
> developed; I think it was actually David (Reiter) who asked on the git
> list about bzr-to-git syncing tools, and IIRC, the replies didn't seem
> so encouraging.

Since converting CVS to Git seems fundamentally more difficult than Bzr
to Git, and seeing the amount of effort that was put into getting a good
Git mirror, I'm fairly confident someone will come up with a good Git
mirror for the Bzr repository.

> [Another problem is that even if another syncing tool is available,
> I'm not sure what changing to another source-VCS/tool means for the
> existing history in git...]

Actually, since the Bzr repository will actually be a conversion of the
Git repository, that should not be a problem (at least in theory).
Especially since AFAIK Bzr keeps a superset of the info kept in Git, so
the roundtrip Git→Bzr→Git should be naturally lossless.


        Stefan

Re: Bazaar migration status?

[Andreas Schwab]

Miles Bader <miles@gnu.org> writes:

> [Another problem is that even if another syncing tool is available,
> I'm not sure what changing to another source-VCS/tool means for the
> existing history in git...]

Most likely the history will come out different if you recreate the git
tree based on the bzr tree, but that's not a big problem.  If you
maintain any local changes on top of the old git tree just rebase them
on top of the new tree.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Bazaar migration status?

[Ken Raeburn]

On Jul 21, 2009, at 04:19, Andreas Schwab wrote:
> Most likely the history will come out different if you recreate the  
> git
> tree based on the bzr tree, but that's not a big problem.  If you
> maintain any local changes on top of the old git tree just rebase them
> on top of the new tree.

And if we're publishing (or planning to publish) git repositories with  
development work based on the Emacs one?  Rebasing is strongly  
discouraged for public repositories.  I'm not all that worried...  
sadly, this wouldn't be the first time VCS lossage has caused me to  
scrap the history of my little project and start again from my latest  
source tree. :-(

I'm not at all familiar with bzr or the conversion tools, but I wonder  
(somewhat idly) if it's possible to rebase within the savannah mirror  
or give it existing revisions as starting points and mirror only newer  
stuff on top of the old mirror... aside from needing to figure out  
what the mirroring scripts are keeping track of and adjust that data,  
it seems like basically the same work as you're suggesting anyone  
using git downstream should do for themselves.  (More work, but not  
repeated across however many people with varying levels of expertise.   
On the other hand, if the mirror ever needed reconstructing from  
scratch, it might not be easily reproducible.)

Ken

Re: Bazaar migration status?

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 1114 bytes --]

On Jul 22, 2009, at 12:31 AM, Ken Raeburn wrote:
>
> And if we're publishing (or planning to publish) git repositories  
> with development work based on the Emacs one?  Rebasing is strongly  
> discouraged for public repositories.

Indeed.  I also don't see how tags (e.g. versions) are supposed to  
work or be recreated if you just rebase several months worth of  
changes as if they happened much later.

I would think that rebasing also gets very complicated if changes are  
interleaved so that patches don't apply cleanly on top of a single  
revision.

 >give it existing revisions as starting points and mirror only newer  
stuff on top of the
 >old mirror... aside from needing to figure out what the mirroring  
scripts are keeping >track of and adjust that data, it seems like  
basically the same work as you're >suggesting anyone using git  
downstream should do for themselves.

Exactly.

> On the other hand, if the mirror ever needed reconstructing from  
> scratch, it might not be easily reproducible.)

I think that's unlikely - rather one would go back to a backup in the  
case of corruption.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 2193 bytes --]

Re: Bazaar migration status?

[Andreas Schwab]

Ken Raeburn <raeburn@raeburn.org> writes:

> On Jul 21, 2009, at 04:19, Andreas Schwab wrote:
>> Most likely the history will come out different if you recreate the git
>> tree based on the bzr tree, but that's not a big problem.  If you
>> maintain any local changes on top of the old git tree just rebase them
>> on top of the new tree.
>
> And if we're publishing (or planning to publish) git repositories with
> development work based on the Emacs one?  Rebasing is strongly
> discouraged for public repositories.

I was only talking about a one-off rebasing, when migrating from the
current git mirror based on the cvs repo to a future git mirror based on
the bzr repo.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Bazaar migration status?

[Ken Raeburn]

On Jul 22, 2009, at 17:33, Andreas Schwab wrote:
>> And if we're publishing (or planning to publish) git repositories  
>> with
>> development work based on the Emacs one?  Rebasing is strongly
>> discouraged for public repositories.
>
> I was only talking about a one-off rebasing, when migrating from the
> current git mirror based on the cvs repo to a future git mirror  
> based on
> the bzr repo.

The advice I've seen isn't "don't do it too often", it's "don't do  
it".  The recovery process seems to be a manual one for people  
downstream from those rebasing.  But, I think I can find a way to turn  
it into a "merge" from one history to another for most of my more  
interesting branches.  It might not be *too* horrible for downstream.

(For that matter, I was "only" talking about a one-off project to make  
the new bzr-based git history at savannah look as though it was  
continuing on from the old cvs-based git history.)

Ken

Re: Bazaar migration status?

[Stephen J. Turnbull]

Ken Raeburn writes:

 > The advice I've seen isn't "don't do it too often", it's "don't do  
 > it".

Rebase a copy of the branch instead of the original, if it worries you
so much:

    git checkout -b rebased-branch original-branch
    git rebase master

If you're worried about this, but have no control over the main repo,
there's always

    # proof of concept; may fail spectacularly if you have defaults
    # configured
    git-my-pull () { git branch save-branch; git pull $@; }

Advocacy:

True, you will see "don't ever rebase" a lot from git-phobes.  It
should be considered to be a litmus test for git-phobia, not as
reflecting VCS reality.

In fact, rebasing is such an important operation that *all* of the
dVCSes have been forced by user demand to provide it, and *only git*
provides facilities (ie, the reflog) for recovery from an ill-advised
rebase that can be used by newbies.[1]  AFAIK, all of hg, bzr, and
darcs can *destroy* some DAG data basically irrecoverably, while git
*always* preserves it (until the next git-gc --prune, which by default
is so far in the future that it's of the same order of magnitude as an
Emacs release cycle ... uh, well, I exaggerate, but you get the
point).

Git-advocate-who-has-helped-advise-two-transitions-to-hg-ly y'rs,

Footnotes: 
[1]  Admittedly, untangling a git-rebase is not for wilting violet
newbies, but anybody who has learned to accept dying in Nethack should
be able to handle it -- it's exactly the same experience: you didn't
want it to happen, it will take time and effort to get back to where
you were, but no permanent harm is ever done.

Re: Bazaar migration status?

[Miles Bader]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> Ken Raeburn writes:
>  > The advice I've seen isn't "don't do it too often", it's "don't do  
>  > it".
>
> True, you will see "don't ever rebase" a lot from git-phobes.  It
> should be considered to be a litmus test for git-phobia, not as
> reflecting VCS reality.

I agree.  Rebasing is a tool; it's often both useful and the "right
thing to do".  Obviously it's good to understand the implications before
doing so, of course.  If someone doesn't understand them, then sure,
they can avoid rebasing until they feel more confident.

That seems to be the reason why git feels so much better than other
dvcses to me -- it tries to be a flexible toolbox, and while it makes
some methods of operations very convenient, it also tries to avoid
unnecessary limitations, and has safety mechanisms that are useful even
when you're doing really wacky stuff.  Other vcses on the other hand,
often seem to start with a simple workflow adequate for common
situations, and adopt that workflow as dogma (and of course, sometimes
embed it into the implementation to the point where it becomes
self-fulfilling).

-Miles

-- 
Philosophy, n. A route of many roads leading from nowhere to nothing.

Re: Bazaar migration status?

[Ken Raeburn]

On Jul 22, 2009, at 23:49, Stephen J. Turnbull wrote:
> True, you will see "don't ever rebase" a lot from git-phobes.  It
> should be considered to be a litmus test for git-phobia, not as
> reflecting VCS reality.

I may have overstated it a little, but the general recommendation is  
that it's a bad idea and should be avoided if possible.  For example,  
"git help rebase" tells me so:

        Rebasing (or any other form of rewriting) a branch that others  
have based
        work on is a bad idea: anyone downstream of it is forced to  
manually fix
        their history. This section explains how to do the fix from the
        downstream's point of view. The real fix, however, would be to  
avoid
        rebasing the upstream in the first place.

This sort of ripple effect requiring manual intervention for everyone  
downstream seems... rude.

> In fact, rebasing is such an important operation that *all* of the
> dVCSes have been forced by user demand to provide it,

I'd believe it, and certainly its use in private branches makes sense  
to me.  My experience with dVCSes is minimal so far, but I can imagine  
a case where some sort of upstream repository change or rebuild is  
absolutely necessary and it's impossible to keep the content the  
same.  In our case, though, it seems to be that we *do* want the  
content to be the same, and all the history info... but we're going to  
have a complete new version history anyways, because it's how the  
tools work today.

Ken

Re: Bazaar migration status?

[Stephen J. Turnbull]

Ken Raeburn writes:

 > I may have overstated it a little, but the general recommendation is  
 > that it's a bad idea and should be avoided if possible.  For example,  
 > "git help rebase" tells me so:
 > 
 >         Rebasing (or any other form of rewriting) a branch that
 >         others have based work on is a bad idea:

"Based work on" is the key phrase.  If you *know* nobody has done so,
if you know who has done so and coordinate with them, it's no big
deal, and often the best way to proceed because everybody agrees that
the branch should be rebased, it's just a question of coordinating on
*when*.

In these cases, fixing is a no-op (if there are no downstream users)
or often trivial (just rebase your working branch using the three-head
form of rebase).  And it's cheap to try, just create a temporary
branch or tag to record the head of the branch you've been working on.

Note that eventually you *are* going to have to rebase, too, because
the mainline *will* proceed to use a merged branch even if they rename
it instead of rebasing it.  git can (and now does) automatically warn
you about the situation if there is a rebase: you'll be told that you
can't pull because it's not a fast forward.

It's not clear to me that this is a bad thing.  The alternative is
that upstream creates a new branch to avoid rebasing issues, and some
weeks later you realize that you haven't pulled any new content for
some weeks, and discover that not only have they rebased into conflict
with your work, but also the new branch itself contains a bunch of new
conflicts that you have to catch up to.  Sure, you could carefully
follow the mailing list(s) to find announcements of branches related
to the one you're based off of, but IMHO this is the kind of thing I'd
like to delegate to my VCS.

 >         anyone downstream of it is forced to manually fix their
 >         history. This section explains how to do the fix from the
 >         downstream's point of view. The real fix, however, would be
 >         to avoid rebasing the upstream in the first place.
 > 
 > This sort of ripple effect requiring manual intervention for everyone  
 > downstream seems... rude.

If done without coordination, sure, it's rude.  Somebody who's done
the amount of work that Andreas has done to preserve history is
anything but likely to be rude, though.

 > [I]t seems to be that we *do* want the content to be the same, and
 > all the history info... but we're going to have a complete new
 > version history anyways, because it's how the tools work today.

Hm?  Uh, if the content is the same, then you haven't rebased at all.
In that case, somebody has deliberately changed the metadata of
history (eg, git-filter-branch).  But you can analyze this with the
tools git (uniquely, AFAIK, cf Miles's "Tribute to a VCS" post :-)
provides:

    find-tree () { git log --all --pretty='%T %H' | grep ^$1; }

which you probably want to invoke as

    find-tree `git cat-file commit HEAD | grep ^tree | cut -b6-`

which will find out if there's any commit whose tree is identical to
HEAD's.  Note that if this is the case, you probably can find a whole
branch which is identical to HEAD's branch in terms of content.


Footnotes: 
[1]  I haven't tested it properly, but that's my reading of the docs
for "git-fetch -f".

Re: Bazaar migration status?

[Ken Raeburn]

On Jul 23, 2009, at 04:14, Stephen J. Turnbull wrote:
> "Based work on" is the key phrase.  If you *know* nobody has done so,
> if you know who has done so and coordinate with them, it's no big
> deal, and often the best way to proceed because everybody agrees that
> the branch should be rebased, it's just a question of coordinating on
> *when*.

Yeah, and it'll probably be easier than I think in reality, because  
while my repository is in theory public, I haven't told anyone where  
it is yet.  But I had been planning to, and wanting to do it soon.   
And distributed development like git (or bzr) supports shouldn't  
require that you keep track of everyone downstream from you.

But between the indefinite timing of the bzr switchover, and the  
warnings about how bad rebasing can be for anyone downstream, the  
message I'm hearing sounds a little like "if you publicize your  
development repo, we will cause you pain." :-)

>>        anyone downstream of it is forced to manually fix their
>>        history. This section explains how to do the fix from the
>>        downstream's point of view. The real fix, however, would be
>>        to avoid rebasing the upstream in the first place.
>>
>> This sort of ripple effect requiring manual intervention for everyone
>> downstream seems... rude.
>
> If done without coordination, sure, it's rude.  Somebody who's done
> the amount of work that Andreas has done to preserve history is
> anything but likely to be rude, though.

I didn't mean to cast aspersions on anyone working on this.  I guess  
I'm just wishing the tools provided some extra metadata were carried  
along somewhere that would make it Just Work without me having to  
track down and sort out where the two versions of history match up.   
(And likewise for everyone downstream.)  After all, it will be in a  
sense the same repository after the conversion as it was before;  
tracking it across the switch ideally ought to be trivial and automatic.

>> [I]t seems to be that we *do* want the content to be the same, and
>> all the history info... but we're going to have a complete new
>> version history anyways, because it's how the tools work today.
>
> Hm?  Uh, if the content is the same, then you haven't rebased at all.

I'm assuming we want the history as shown by git or bzr to show  
exactly the same sources, changed in exactly the same way at exactly  
the same times and by the same people and with the same log messages,  
as we see now with CVS or git.

What I've been hearing is that the result of mirroring the bzr  
repository into git isn't likely to give the same commit revision ids  
as we have in the CVS->git mirror.  Which suggests to me that either  
the content or the metadata is going to come out differently in some  
way, probably because of the differences between tools.  (Or maybe the  
mirroring tools are non-deterministic in some way affecting the  
output, but I would hope that wouldn't be the case.)  If the bzr->git  
mirror were expected to generate the same hash values for absolutely  
everything, I wouldn't have anything to be concerned about.

> In that case, somebody has deliberately changed the metadata of
> history (eg, git-filter-branch).  But you can analyze this with the
> tools git (uniquely, AFAIK, cf Miles's "Tribute to a VCS" post :-)
> provides:
[...]
> which will find out if there's any commit whose tree is identical to
> HEAD's.  Note that if this is the case, you probably can find a whole
> branch which is identical to HEAD's branch in terms of content.

Interesting... that could be the additional data I was asking for  
above.  Thanks!
But, the expectation of different revision ids still concerns me -- is  
that because of minor differences in content, like maybe  
translating .cvsignore into something else for bzr or loss of some  
trivial bit of data like file modes, that will cause the trees not to  
be identical, or differences in commit metadata that just cause the  
commit ids to be different despite matching file content?

Ken

Re: Bazaar migration status?

[Stephen J. Turnbull]

Ken Raeburn writes:

 > I didn't mean to cast aspersions on anyone working on this.

Sorry, I didn't mean to imply you did, just that this is a one-time
thing and I think we can assume that the workers will give adequate
warning and documentation.

 > I guess I'm just wishing the tools provided some extra metadata
 > were carried along somewhere that would make it Just Work without
 > me having to track down and sort out where the two versions of
 > history match up.

Unfortunately, defining and computing "two versions of history match
up" is extremely hard.  Humans look at a program and say "obviously
this change is irrelevant to anything I care about."  But to the VCS,
a change is a change.  For example, ChangeLogs are a real PITN.
Suppose you change Emacs (patch A), and I change Emacs (patch B).  Now
we mutually cherrypick and merge.  In a DAG-based VCS (which can
represent concurrent development of this kind), we should be in the
same state, right?  Wrong.  The DAG can represent non-determinism, but
*the ChangeLog cannot*.  There's no reason to expect our trees are
identical, and every reason to expect them to diverge (you probably
cherrypick mine on top of yours, and I do the reverse, so our change
logs have different orders).

Now, if we coordinate our workflow, we can probably get it right: I
pull from you and you pull from me, and whoever pulls first will
determine the order.  But of course we're assuming here uncoordinated
workflows.

 > (And likewise for everyone downstream.)  After all, it will be in a  
 > sense the same repository

To quote one of my less favorite Presidents, "There you go again!" :-)
See?  You're very willing to go with "I know they're the same, so, yo
git, what's your problem?"  I have no problems with humans saying
that, but it's the *last* kind of attitude we want our VCS to take!

 > after the conversion as it was before; tracking it across the
 > switch ideally ought to be trivial and automatic.

It is, except that the conversion from CVS is fraught with
nondeterminism; that's why the work that Andreas is doing is so
important.  After that, all of the dVCSes are happy to work with git's
fastimport format so in that sense they can share history.  (AFAIK, to
compute revision IDs they all use a subset of git's revision-ID-
relevant metadata: name, email, and timestamp for committer and author
respectively, tree content -- perhaps represented as a digest, and log
message.  There are variations, but so far they're all homomorphic --
I think git actually keeps the most detail.  I wish git had adopted a
Dublin Core derivative as a framework for metadata extensions, though
... the lack of a comprehensive standard here makes me nervous.)

 > I'm assuming we want the history as shown by git or bzr to show  
 > exactly the same sources, changed in exactly the same way at exactly  
 > the same times and by the same people and with the same log messages,  
 > as we see now with CVS or git.

It will, up to the limitations of CVS.  Commit timestamps may differ
slightly from CVS (the official cvs2svn tool will amend CVS commit
timestamps to ensure that the revision numbers and timestamps are the
same total order, compatible with the ancestry partial order -- I
don't know if that's what Andreas is using, though).  Of course, CVS
timestamps are in general ranges (each file commit gets a separate
timestamp).

 > What I've been hearing is that the result of mirroring the bzr  
 > repository into git isn't likely to give the same commit revision ids  
 > as we have in the CVS->git mirror.

That, I believe, is true.  But AIUI it's not because of inherent
nondeterminism in the conversion process being used, it's because CVS
suffers from nonatomic commits.  The history is continuously being
cleaned up (the conversion process tuned), so that in fact the
official bzr repository will have a different history from the
existing CVS->git mirror.

However, the content is going to be "very close" to the current git
mirror.  I have some ideas how to exploit that for a conversion of
your git repo; feel free to get in touch with me off-list.  (Once you
have a git repo, the git->bzr conversion is straightforward.)

Note that in git you can guarantee that your branch gets imported
lock, stock and barrel with *zero* content changes by using
git-filter-branch.  The idea is to reset the node of your branch to be
somewhere in the new tree by changing its parent.  This will result in
new revision IDs for all commits in the branch, of course, but the
content and all metadata (except for ancestry) is identical.

You can even just import your whole repo into your clone of the
"official" git mirror of the Emacs repo; git itself doesn't care.
Although this is likely to confuse your downstream (and maybe even
you), as well as tools like gitk, because the repo will be
multi-rooted.  But you can then use all the git tools (rebase,
filter-branch, etc) to work with the resulting (disconnected) DAG.
You can even reconnect it, although it's not likely that the result
will be much more useful than the disconnected DAG.  (Look for "paste
the other history" in git-filter-branch(1) for details.)

 > > In that case, somebody has deliberately changed the metadata of
 > > history (eg, git-filter-branch).  But you can analyze this with the
 > > tools git provides: [...].

 > Interesting... that could be the additional data I was asking for  
 > above.  Thanks!
 > But, the expectation of different revision ids still concerns me -- is  

Well, you can also use git-log --diff-stat to get information that can
be used to find "closest approach".  (This is the idea I mentioned above.)

 > that because of minor differences in content, like maybe  
 > translating .cvsignore into something else for bzr or loss of some  
 > trivial bit of data like file modes,

The different VCSes calculate the revision ID differently, but there
should be 1-1 conversion (up to the possibility of hash collisions, of
course).  But the conversion process is automatic and metadata
oriented; it will not convert content.  To some extent that can be
done automatically, but it should be done as a commit on top of the
converted repo.

BTW, don't get concerned about all the potential complexity I'm
describing.  You'll only need it once, and there are a lot of gitfans
around who will be more than happy to help with it.  We'll even look
the other way if you decide to convert everything to a bzr repo in the
end. :-)  Ie, git is the tool of choice for doing the surgery on
history, but after that, it's personal preference (and the conversion
processes among the leading dVCSes are straightforward).

Re: Bazaar migration status?

[Ken Raeburn]

On Jul 23, 2009, at 22:43, Stephen J. Turnbull wrote:
>> (And likewise for everyone downstream.)  After all, it will be in a
>> sense the same repository
>
> To quote one of my less favorite Presidents, "There you go again!" :-)
> See?  You're very willing to go with "I know they're the same, so, yo
> git, what's your problem?"  I have no problems with humans saying
> that, but it's the *last* kind of attitude we want our VCS to take!

In the general case I'd agree, but here we're talking about two  
conversions of the same repository -- one from cvs to git, and the  
other from cvs to bzr and then to git.

> It is, except that the conversion from CVS is fraught with
> nondeterminism;

Ah, I guess that's what I was missing....

> That, I believe, is true.  But AIUI it's not because of inherent
> nondeterminism in the conversion process being used, it's because CVS
> suffers from nonatomic commits.  The history is continuously being
> cleaned up (the conversion process tuned), so that in fact the
> official bzr repository will have a different history from the
> existing CVS->git mirror.

That makes sense.  Thank you for the explanation.  I think I was  
assuming we'd have the same conversion process, without any tuning  
since whatever generated the git repo.  But given that we're talking  
about conversion of the main, official repository, getting it done as  
well as possible this time makes more sense.

Ken

Re: Bazaar migration status?

[Stephen J. Turnbull]

Ken Raeburn writes:

 > conversions of the same repository -- one from cvs to git, and the  
 > other from cvs to bzr and then to git.

AIUI, that conversion is going to be cvs -> git/fastimport -> bzr. :-)

Re: Bazaar migration status?

[Andreas Schwab]

Ken Raeburn <raeburn@raeburn.org> writes:

> The advice I've seen isn't "don't do it too often", it's "don't do
> it".

We are not talking about an official main repository, but a secondary
"nice to have" one.  You will never ever see Linus to rewind its
repository once he has published it, but several other kernel
repositories that are the development platform of certain parts of the
kernel get rebased pretty regularily (usually following the release of a
new kernel version).  The linux-next repository is even rebased daily.

> (For that matter, I was "only" talking about a one-off project to make
> the new bzr-based git history at savannah look as though it was
> continuing on from the old cvs-based git history.)

Trying to achieve that continuity is likely much more effort than to
just ask all users to rebase once.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Bazaar migration status?

[Christian Faulhammer]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

Hi,

Chong Yidong <cyd@stupidchicken.com>:

> joakim@verona.se writes:
> 
> >> But, yes, I'm also eager to see us move away from CVS.
> >
> > Can we translate this mutual eagerness into actual action?
> >
> > FWIW when I encounter this situation in my professional role I
> > usually just disable committing to the old repos, leaving it
> > on-line, and start fresh with the new repos.
> 
> We're supposedly waiting for the latest Bazaar version to stabilize
> first, see

 1.17 is out now which marks 2a repository format as "stable".

V-Li

-- 
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode

<URL:http://gentoo.faulhammer.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]