git transition issues

git transition issues

[Eric S. Raymond]

I'm in the process of preparing a transition-day patch series.  These will be
the first patches applied to the new git repository after conversion;
the objective is to replace bzr recipes with with git recipes where
applicable.

For most of the files this patch touches the changes are small and
relatively straightforward. The easy changes are:

      Makefile.in
      etc/CONTRIBUTE
      admin/make-tarball.txt
      admin/update-copyright
      admin/notes/bugtracker
      admin/notes/tags
      admin/update_autogen

Makefile.in, for example, needs to have the value of VCSWITNESS modified

There are three files for which the right thing is not obvious.

admin/bzrmerge.el, admin/notes/repo: bzrmerge.el won't work in the new
repository at all, as it felies on bzr CLI features.  There is
material in admin/notes/repo tied to it.  All this will have to be
rewritten for git. 

lisp/version.el: The bzr-specific code for version extraction will be
obsolete.  The git-specific code is very broken.  Among other
problems, in my stock Ubuntu environment the git log command it
generates always fails, apparently because it can't find terminal
capabilities it wants. 

Bearing in mind that this stuff will remain in the archival bzr repo,
my inclination is to just delete it.  Nothing in the rest of Emacs
uses either .el file. I think it better for user attempts to employ
these to fail loudly so they will be fixed rather than silently
returning incorrect results.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

"America is at that awkward stage.  It's too late to work within the system,
but too early to shoot the bastards."	-- Claire Wolfe

Re: git transition issues

[Stefan Monnier]

> admin/bzrmerge.el, admin/notes/repo: bzrmerge.el won't work in the new
> repository at all, as it felies on bzr CLI features.  There is
> material in admin/notes/repo tied to it.  All this will have to be
> rewritten for git.

Yes, this needs to be rewritten.  I thought it was done already.  It was
mentioned as one of the prerequisites for a move to Git, since we use it
on a regular basis to merge the new emacs-24 changes into trunk.


        Stefan

Re: git transition issues

[Eric S. Raymond]

Stefan Monnier <monnier@iro.umontreal.ca>:
> Yes, [bzrmerge.el] needs to be rewritten.  I thought it was done
> already.  It was mentioned as one of the prerequisites for a move to
> Git, since we use it on a regular basis to merge the new emacs-24
> changes into trunk.

The prerequisite I understood - and satisfied, I believe - was to add
the vc-git.el support (equivalent to certain methods in the bzr back
end) required to rewrite this code.

I can't rewrite the upper level myself because I am not familiar with
the workflow it was designed to support.  To the very limited extent I
understand the bzr-merge.el code, I'm not even sure its assumptions 
make sense anymore - git has a sufficently different model of branches
and merging to make that questionable.

I would be quite willing to tackle this if I were qualified to do so,
but the code will need to be rewritten and rethought by a maintainer
much more familiar with past practice and its rationale than I am.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

On 10/26/14 5:07 PM, esr@thyrsus.com (Eric S. Raymond) wrote:

> lisp/version.el: The bzr-specific code for version extraction will be
> obsolete.  The git-specific code is very broken.  Among other
> problems, in my stock Ubuntu environment the git log command it
> generates always fails, apparently because it can't find terminal
> capabilities it wants.

From version.el:
			  (call-process "git" nil '(t nil) nil "log"
					"-1" "--pretty=format:%N")

This looks wrong to me. %N is the commit's note from "git notes".
Perhaps this should be %H, to get the commit hash? And if so, I think
the better way is with "git show-ref", as it's a plumbing command where
"git log" is not (git log likes to redirect stuff through your $PAGER, etc).

  git show-ref -s --head ^HEAD

-David



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]

Re: git transition issues

[Eric S. Raymond]

David Caldwell <david@porkrind.org>:
> On 10/26/14 5:07 PM, esr@thyrsus.com (Eric S. Raymond) wrote:
> 
> > lisp/version.el: The bzr-specific code for version extraction will be
> > obsolete.  The git-specific code is very broken.  Among other
> > problems, in my stock Ubuntu environment the git log command it
> > generates always fails, apparently because it can't find terminal
> > capabilities it wants.
> 
> From version.el:
> 			  (call-process "git" nil '(t nil) nil "log"
> 					"-1" "--pretty=format:%N")
> 
> This looks wrong to me. %N is the commit's note from "git notes".
> Perhaps this should be %H, to get the commit hash? And if so, I think
> the better way is with "git show-ref", as it's a plumbing command where
> "git log" is not (git log likes to redirect stuff through your $PAGER, etc).
> 
>   git show-ref -s --head ^HEAD

I noticed the %N issue, yes.  I agree with this critique.

Consider this change, part of my transition-day patch:

+# Ideally, VCSWITNESS should be a file that is modified whenever the
+# repository registers a commit from either a local checkin or a
+# repository pull. In git there is no single file that guarantees
+# this, but the local log for the current head should be close enough.
+#
 # Note the use of single quotes in the value of vcswitness.
 # This passes an unexpanded $srcdir to src's Makefile, which then
 # expands it using its own value of srcdir (which points to the
 # source directory of src/).
 src: Makefile
-	dirstate='.bzr/checkout/dirstate';				\
+	dirstate='.git/logs/HEAD';				\
 	vcswitness='$$(srcdir)/../'$$dirstate;				\
 	[ -r "$(srcdir)/$$dirstate" ] || vcswitness='';			\
 	$(MAKE) -C $@ all VCSWITNESS="$$vcswitness"

Can you think of a better witness value?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[Stefan Monnier]

> the workflow it was designed to support.  To the very limited extent I
> understand the bzr-merge.el code, I'm not even sure its assumptions 
> make sense anymore - git has a sufficently different model of branches
> and merging to make that questionable.

There's no particular assumption made in bzrmerge.el about the model
of branching.  Al it does is "merge the branch", just with tweaks to
avoid spurious conflicts.

I.e. it first gets the list of revisions that will be brought in by the
merge (this is a linear list of commits, if one of those commits is
a merge commit, then the only part that shows is this commit, not the
"subcommits" that were merged), then splits it into those commits that
are backports (or similar, recognized by special keywords in the commit
message) and who hence have already been merged (they'll get merged by
"bzr merge; bzr revert" so the only thing merged is the meatadata) and
the others which are merged in the normal way.

In DaRCS, it would work differently, but Git and Bzr have exactly the
same branching model in this respect.


        Stefan

Re: git transition issues

[Glenn Morris]

"Eric S. Raymond" wrote:

>> This looks wrong to me. %N is the commit's note from "git notes".
>> Perhaps this should be %H, to get the commit hash? And if so, I think
>> the better way is with "git show-ref", as it's a plumbing command where
>> "git log" is not (git log likes to redirect stuff through your $PAGER, etc).
>> 
>>   git show-ref -s --head ^HEAD
>
> I noticed the %N issue, yes.  I agree with this critique.

%N is right for the current git repo.

http://lists.gnu.org/archive/html/emacs-devel/2014-08/msg00442.html

Re: git transition issues

[Glenn Morris]

"Eric S. Raymond" wrote:

> Stefan Monnier <monnier@iro.umontreal.ca>:
>
>> Yes, [bzrmerge.el] needs to be rewritten.  I thought it was done
>> already.  It was mentioned as one of the prerequisites for a move to
>> Git, since we use it on a regular basis to merge the new emacs-24
>> changes into trunk.
>
> The prerequisite I understood - and satisfied, I believe - was to add
> the vc-git.el support (equivalent to certain methods in the bzr back
> end) required to rewrite this code.

I mentioned bzrmerge.el needing to be replaced more than once, eg

http://lists.gnu.org/archive/html/emacs-devel/2014-09/msg00302.html

Re: git transition issues

[Eric S. Raymond]

Glenn Morris <rgm@gnu.org>:
> I mentioned bzrmerge.el needing to be replaced more than once

I'm sure you did. That doesn't mean I can do it, or should even try.

It needs to be replaced by someone who has used it and *knows how it
should work*.  That isn't me.

It is reasonable to expect that if something in the rewrite makes vc-git.el
go sproing I will fix that.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[Eli Zaretskii]

Btw, all these changes are being committed to the trunk, but the
release branch doesn't have them.  However, if everything goes
according to plan, Emacs 24.5 is supposed to be made from the release
branch _after_ the transition to git.  So I think at least most, if
not all, of these changes need to be committed to the branch, and then
merged to the trunk as usual.  (For those changes that were already
committed to the trunk, they should be cherry-picked to the branch.)

I hope no one is entertaining the idea of having live bzr and git
repos in parallel, one for the release branch, the other for the
trunk.  Right?

Re: git transition issues

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> Btw, all these changes are being committed to the trunk, but the
> release branch doesn't have them.  However, if everything goes
> according to plan, Emacs 24.5 is supposed to be made from the release
> branch _after_ the transition to git.  So I think at least most, if
> not all, of these changes need to be committed to the branch, and then
> merged to the trunk as usual.  (For those changes that were already
> committed to the trunk, they should be cherry-picked to the branch.)

If that is telling ne to do any further peparatory commits to the
24.5 release branch, I will attempt to comply.
 
> I hope no one is entertaining the idea of having live bzr and git
> repos in parallel, one for the release branch, the other for the
> trunk.  Right?

Oh Goddess, no.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[Eli Zaretskii]

> Date: Tue, 28 Oct 2014 14:07:39 -0400
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org>:
> > Btw, all these changes are being committed to the trunk, but the
> > release branch doesn't have them.  However, if everything goes
> > according to plan, Emacs 24.5 is supposed to be made from the release
> > branch _after_ the transition to git.  So I think at least most, if
> > not all, of these changes need to be committed to the branch, and then
> > merged to the trunk as usual.  (For those changes that were already
> > committed to the trunk, they should be cherry-picked to the branch.)
> 
> If that is telling ne to do any further peparatory commits to the
> 24.5 release branch, I will attempt to comply.

I think so, but let's first hear from Stefan and Glenn; perhaps I'm
missing something.

Re: git transition issues

[Randal L. Schwartz]

>>>>> "David" == David Caldwell <david@porkrind.org> writes:

David> On 10/26/14 5:07 PM, esr@thyrsus.com (Eric S. Raymond) wrote:
>> lisp/version.el: The bzr-specific code for version extraction will be
>> obsolete.  The git-specific code is very broken.  Among other
>> problems, in my stock Ubuntu environment the git log command it
>> generates always fails, apparently because it can't find terminal
>> capabilities it wants.

David> From version.el:
David> 			  (call-process "git" nil '(t nil) nil "log"
David> 					"-1" "--pretty=format:%N")

David> This looks wrong to me. %N is the commit's note from "git notes".
David> Perhaps this should be %H, to get the commit hash? And if so, I think
David> the better way is with "git show-ref", as it's a plumbing command where
David> "git log" is not (git log likes to redirect stuff through your $PAGER, etc).

David>   git show-ref -s --head ^HEAD

Consider instead something like "git describe --tags HEAD", which
generates a fairly short yet meaningful and distinct value regardless of
where HEAD is.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix consulting, Technical writing, Comedy, etc. etc.
Still trying to think of something clever for the fourth line of this .sig

Re: git transition issues

[Randal L. Schwartz]

>>>>> "Randal" == Randal L Schwartz <merlyn@stonehenge.com> writes:

Randal> Consider instead something like "git describe --tags HEAD", which
Randal> generates a fairly short yet meaningful and distinct value regardless of
Randal> where HEAD is.

That pulls in too many tags.  This is closer:

  git describe --tags --match "EMACS_2*" HEAD

which will break when emacs version 30 comes out, but it's good enough
for now.  For example, for the current origin/emacs-24 commit, that
comes out as:

  EMACS_23_4-22895-g6b5af2d

which says we're 22895 commits past EMACS_23_4 in the repo, and the last
8 chars are "g" plus the head of the sha1 (6b5af2d), uniquely
identifying the commit.

If the head is precisely the tag, the output is just the tag, so for:

  git describe --tags --match "EMACS_2*" EMACS_23_4

we get

  EMACS_23_4

I think this has the properties you want, although the first part might
require massaging.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix consulting, Technical writing, Comedy, etc. etc.
Still trying to think of something clever for the fourth line of this .sig

Re: git transition issues

[Stefan Monnier]

> I think so, but let's first hear from Stefan and Glenn; perhaps I'm
> missing something.

No, I think you're absolutely right.  Maybe not all the changes need to
be backported, but many of them should.  I recommend to backport
most/all of them as a single commit.


        Stefan

Re: git transition issues

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 28 Oct 2014 20:58:33 -0400
> Cc: esr@thyrsus.com, emacs-devel@gnu.org
> 
> No, I think you're absolutely right.  Maybe not all the changes need to
> be backported, but many of them should.  I recommend to backport
> most/all of them as a single commit.

I ended up backporting all of them, but in 3 different commits.  A few
unrelated commits were done in-between Eric's, and since bzr doesn't
by default allow merging when there are uncommitted changes (and I
didn't want to use --force to override that), I couldn't easily commit
all the cherry-picked revisions in one go.  But I think 3 is not that
bad.  (The 3rd one is Glenn's revert of one part of Eric's commits.)

Re: git transition issues

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Date: Tue, 28 Oct 2014 20:58:33 -0400
> > Cc: esr@thyrsus.com, emacs-devel@gnu.org
> > 
> > No, I think you're absolutely right.  Maybe not all the changes need to
> > be backported, but many of them should.  I recommend to backport
> > most/all of them as a single commit.
> 
> I ended up backporting all of them, but in 3 different commits.  A few
> unrelated commits were done in-between Eric's, and since bzr doesn't
> by default allow merging when there are uncommitted changes (and I
> didn't want to use --force to override that), I couldn't easily commit
> all the cherry-picked revisions in one go.  But I think 3 is not that
> bad.  (The 3rd one is Glenn's revert of one part of Eric's commits.)

Hm.  Which target branch are we talking?  Because I ported those changes
to the emacs-24 branch yesterday.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[Eli Zaretskii]

> Date: Fri, 31 Oct 2014 06:23:22 -0400
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> > I ended up backporting all of them, but in 3 different commits.  A few
> > unrelated commits were done in-between Eric's, and since bzr doesn't
> > by default allow merging when there are uncommitted changes (and I
> > didn't want to use --force to override that), I couldn't easily commit
> > all the cherry-picked revisions in one go.  But I think 3 is not that
> > bad.  (The 3rd one is Glenn's revert of one part of Eric's commits.)
> 
> Hm.  Which target branch are we talking?

emacs-24, of course.

> Because I ported those changes to the emacs-24 branch yesterday.

I don't see any of that in "bzr log"s output in that branch.  Can you
tell which revision numbers or revision-IDs you committed to emacs-24
yesterday?

Re: git transition issues

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > Date: Fri, 31 Oct 2014 06:23:22 -0400
> > From: "Eric S. Raymond" <esr@thyrsus.com>
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> > 
> > > I ended up backporting all of them, but in 3 different commits.  A few
> > > unrelated commits were done in-between Eric's, and since bzr doesn't
> > > by default allow merging when there are uncommitted changes (and I
> > > didn't want to use --force to override that), I couldn't easily commit
> > > all the cherry-picked revisions in one go.  But I think 3 is not that
> > > bad.  (The 3rd one is Glenn's revert of one part of Eric's commits.)
> > 
> > Hm.  Which target branch are we talking?
> 
> emacs-24, of course.
> 
> > Because I ported those changes to the emacs-24 branch yesterday.
> 
> I don't see any of that in "bzr log"s output in that branch.  Can you
> tell which revision numbers or revision-IDs you committed to emacs-24
> yesterday?

Checking...117637 in my log.  

------------------------------------------------------------
revno: 117637
committer: Eric S. Raymond <esr@thyrsus.com>
branch nick: emacs-24
timestamp: Thu 2014-10-30 15:21:32 -0400
message:
  Port preparatory patches for the git transition from trunk.
  
  These contain no git-specific language or code; rather, they change
  language that was bzr-specific to be independent of the VCS type.

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[Eli Zaretskii]

> Date: Fri, 31 Oct 2014 06:51:42 -0400
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> > > Hm.  Which target branch are we talking?
> > 
> > emacs-24, of course.
> > 
> > > Because I ported those changes to the emacs-24 branch yesterday.
> > 
> > I don't see any of that in "bzr log"s output in that branch.  Can you
> > tell which revision numbers or revision-IDs you committed to emacs-24
> > yesterday?
> 
> Checking...117637 in my log.  
> 
> ------------------------------------------------------------
> revno: 117637
> committer: Eric S. Raymond <esr@thyrsus.com>
> branch nick: emacs-24
> timestamp: Thu 2014-10-30 15:21:32 -0400
> message:
>   Port preparatory patches for the git transition from trunk.
>   
>   These contain no git-specific language or code; rather, they change
>   language that was bzr-specific to be independent of the VCS type.

I guess you didn't push it, then, because revno 117637 in the upstream
repo is this:

  $ bzr log --long -v -c117637 --show-ids
  ------------------------------------------------------------
  revno: 117637
  revision-id: jan.h.d@swipnet.se-20141030191538-3sptx5py8gjdfrw1
  parent: jan.h.d@swipnet.se-20141030172352-wxk72s27bw3rsmic
  fixes bug: http://debbugs.gnu.org/18884
  committer: Jan D. <jan.h.d@swipnet.se>
  branch nick: emacs-24
  timestamp: Thu 2014-10-30 19:15:38 +0000
  message:
    * nsfns.m (ns_set_doc_edited): Do all logic (check frames) here
    instead of in xdisp.c, function now takes void.

    * nsterm.h (ns_set_doc_edited): Declare taking no args.

    * xdisp.c (prepare_menu_bars): Remove HAVE_NS code.
    (redisplay_internal): Call ns_set_doc_edited if HAVE_NS (Bug#18884).
  modified:
    src/ChangeLog                  changelog-20091113204419-o5vbwnq5f7feedwu-1438
    src/nsfns.m                    nsfns.m-20091113204419-o5vbwnq5f7feedwu-8741
    src/nsterm.h                   nsterm.h-20091113204419-o5vbwnq5f7feedwu-8746
    src/xdisp.c                    xdisp.c-20091113204419-o5vbwnq5f7feedwu-240

What does the following command say, when you run it from your
emacs-24 branch?

 $ bzr missing bzr+ssh://esr@bzr.savannah.gnu.org/emacs/emacs-24/

(Assuming your savannah username is esr; if not, change accordingly).
Does this show revno 117637 as missing upstream?

Re: git transition issues

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> What does the following command say, when you run it from your
> emacs-24 branch?
> 
>  $ bzr missing bzr+ssh://esr@bzr.savannah.gnu.org/emacs/emacs-24/
> 
> (Assuming your savannah username is esr; if not, change accordingly).
> Does this show revno 117637 as missing upstream?

Yes, dammit.  Apparently I fat-fingered the commit.  Am now annoyed at
self.  Grrmp.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git transition issues

[Stephen J. Turnbull]

Eric S. Raymond writes:
 > Eli Zaretskii <eliz@gnu.org>:
 > > What does the following command say, when you run it from your
 > > emacs-24 branch?
 > > 
 > >  $ bzr missing bzr+ssh://esr@bzr.savannah.gnu.org/emacs/emacs-24/
 > > 
 > > (Assuming your savannah username is esr; if not, change accordingly).
 > > Does this show revno 117637 as missing upstream?
 > 
 > Yes, dammit.  Apparently I fat-fingered the commit.  Am now annoyed at
 > self.  Grrmp.

Don't be, it's just an occupational hazard with revnos.  *snicker*

Rewriting bzrmerge.el (was: git transition issues)

[David Engster]

Stefan Monnier writes:
>> admin/bzrmerge.el, admin/notes/repo: bzrmerge.el won't work in the new
>> repository at all, as it felies on bzr CLI features.  There is
>> material in admin/notes/repo tied to it.  All this will have to be
>> rewritten for git.
>
> Yes, this needs to be rewritten.  I thought it was done already.  It was
> mentioned as one of the prerequisites for a move to Git, since we use it
> on a regular basis to merge the new emacs-24 changes into trunk.

OK, I'll bite... *sigh*

So, with bzr, we could pretty easily commit only the meta-data of
skipped commits, so that they were regarded as merged. But being the
stupid content tracker that Git is, I think that ship has sailed. We can
of course cherry-pick a commit with the 'ours' merge strategy, but that
will of course change the SHA1...

Ideas?

-David

Re: Rewriting bzrmerge.el

[Paul Eggert]

To be honest, I never used bzrmerge.el the few times that I merged, and I doubt 
whether anybody noticed.  As long as the result was right, and there was some 
indication in the commit log / ChangeLog that it was a merge, people who care 
can figure it out.

More generally, I sense that there's a tension here between having the 
repository reflect the actual history of changes, warts and all, and having the 
repository reflect the history of changes in a nice tidy organized way.  Having 
done a fair bit of software archeology myself I must say that I far prefer the 
former.  (Plus, it's less work for the non-archeologists....)

Re: Rewriting bzrmerge.el

[Stefan Monnier]

> To be honest, I never used bzrmerge.el the few times that I merged, and
> I doubt whether anybody noticed.

Only you could notice, since the point of bzrmerge.el is to auto-resolve
the merge conflicts.


        Stefan

Re: Rewriting bzrmerge.el

[Stefan Monnier]

> So, with bzr, we could pretty easily commit only the meta-data of
> skipped commits, so that they were regarded as merged.  But being the
> stupid content tracker that Git is, I think that ship has sailed.

AFAICT, there is no difference between Git and Bzr in this respect.

> We can of course cherry-pick a commit with the 'ours' merge strategy,
> but that will of course change the SHA1...

bzrmerge.el does not cherry-pick and neither should gitmerge.el.
It should identify those commits that are "backports" (or similar) and
merge them with the `ours' strategy (which will indeed leave the files
unchanged while affecting the metadata, AFAIK).


        Stefan

Re: Rewriting bzrmerge.el

[David Engster]

Stefan Monnier writes:
>> So, with bzr, we could pretty easily commit only the meta-data of
>> skipped commits, so that they were regarded as merged.  But being the
>> stupid content tracker that Git is, I think that ship has sailed.
>
> AFAICT, there is no difference between Git and Bzr in this respect.
>
>> We can of course cherry-pick a commit with the 'ours' merge strategy,
>> but that will of course change the SHA1...
>
> bzrmerge.el does not cherry-pick and neither should gitmerge.el.

OK: It gradually builds up a merge from several small ones using
different strategies.

> It should identify those commits that are "backports" (or similar) and
> merge them with the `ours' strategy (which will indeed leave the files
> unchanged while affecting the metadata, AFAIK).

But after a merge, 'git log emacs-24 ^master' should be empty, just like
'bzr missing ../emacs-24 --theirs-only' was empty, right?  You cannot
achieve that with the 'ours' strategy, since it will not be the same
commit anymore.

-David

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Date: Sat, 15 Nov 2014 15:52:06 +0100
> Cc: "Eric S. Raymond" <esr@thyrsus.com>, emacs-devel@gnu.org
> 
> OK, I'll bite...

Thanks a lot.

> So, with bzr, we could pretty easily commit only the meta-data of
> skipped commits, so that they were regarded as merged. But being the
> stupid content tracker that Git is, I think that ship has sailed.

You mean, if I merge a commit, and then make another commit that
undoes all the changes of the first one, that first commit will not
appear in the DAG?  I'd be surprised.

Or are you saying that if I do the above, then merge again from the
same branch, Git will again bring the same stuff and effectively undo
my undo?

Re: Rewriting bzrmerge.el

[David Engster]

David Engster writes:
> Stefan Monnier writes:
>>> So, with bzr, we could pretty easily commit only the meta-data of
>>> skipped commits, so that they were regarded as merged.  But being the
>>> stupid content tracker that Git is, I think that ship has sailed.
>
>>
>> AFAICT, there is no difference between Git and Bzr in this respect.
>>
>>> We can of course cherry-pick a commit with the 'ours' merge strategy,
>>> but that will of course change the SHA1...
>>
>> bzrmerge.el does not cherry-pick and neither should gitmerge.el.
>
> OK: It gradually builds up a merge from several small ones using
> different strategies.
>
>> It should identify those commits that are "backports" (or similar) and
>> merge them with the `ours' strategy (which will indeed leave the files
>> unchanged while affecting the metadata, AFAIK).
>
> But after a merge, 'git log emacs-24 ^master' should be empty, just like
> 'bzr missing ../emacs-24 --theirs-only' was empty, right?  You cannot
> achieve that with the 'ours' strategy, since it will not be the same
> commit anymore.

On second thought: Do you mean we should just merge them in a series of
merge commits (maybe in a temporary branch as to not pollute 'master'
too much)? That's easy, of course. I was hoping I could somehow
replicate the workflow we did with Bazaar, where we had one single merge
commit for each merge.

-David

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> Date: Sat, 15 Nov 2014 07:40:54 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> To be honest, I never used bzrmerge.el the few times that I merged, and I doubt 
> whether anybody noticed.

If you didn't have to deal with commits that are marked "backported"
or "don't merge to trunk", then the merge is a trivial VCS job, and
indeed doesn't need any help.

Otherwise, bzrmerge.el saves a lot of manual, error-prone typing, no
more, no less.

> More generally, I sense that there's a tension here between having the 
> repository reflect the actual history of changes, warts and all, and having the 
> repository reflect the history of changes in a nice tidy organized way.  Having 
> done a fair bit of software archeology myself I must say that I far prefer the 
> former.  (Plus, it's less work for the non-archeologists....)

I don't think bzrmerge changed anything in this department.

Re: Rewriting bzrmerge.el

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> So, with bzr, we could pretty easily commit only the meta-data of
>> skipped commits, so that they were regarded as merged. But being the
>> stupid content tracker that Git is, I think that ship has sailed.
>
> You mean, if I merge a commit, and then make another commit that
> undoes all the changes of the first one, that first commit will not
> appear in the DAG?  I'd be surprised.
>
> Or are you saying that if I do the above, then merge again from the
> same branch, Git will again bring the same stuff and effectively undo
> my undo?

No. Explicitly reverting the commit is one possibility that works, but
that's ugly and tedious, especially when the commit has conflicts (which
is pretty common, since this is often the reason why it should be
skipped in the first place).

With Bazaar, you could gradually build up your merge in a series of
smaller merges using different strategies (granted, that involved using
the '--force' option, but nonetheless, it was possible). In the end,
this lead to one single merge commit from the emacs-24 branch, which was
nice and tidy. I was hoping one of the git experts around here could
tell me how to do that with git.

-David

Re: Rewriting bzrmerge.el

[David Kastrup]

David Engster <deng@randomsample.de> writes:

> Eli Zaretskii writes:
>>> From: David Engster <deng@randomsample.de>
>>> So, with bzr, we could pretty easily commit only the meta-data of
>>> skipped commits, so that they were regarded as merged. But being the
>>> stupid content tracker that Git is, I think that ship has sailed.
>>
>> You mean, if I merge a commit, and then make another commit that
>> undoes all the changes of the first one, that first commit will not
>> appear in the DAG?  I'd be surprised.
>>
>> Or are you saying that if I do the above, then merge again from the
>> same branch, Git will again bring the same stuff and effectively undo
>> my undo?
>
> No. Explicitly reverting the commit is one possibility that works, but
> that's ugly and tedious, especially when the commit has conflicts (which
> is pretty common, since this is often the reason why it should be
> skipped in the first place).
>
> With Bazaar, you could gradually build up your merge in a series of
> smaller merges using different strategies (granted, that involved using
> the '--force' option, but nonetheless, it was possible). In the end,
> this lead to one single merge commit from the emacs-24 branch, which was
> nice and tidy. I was hoping one of the git experts around here could
> tell me how to do that with git.

git rebase -i

-- 
David Kastrup

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Sat, 15 Nov 2014 18:50:10 +0100
> 
> git rebase -i

Wouldn't that unnecessarily annoy, since most of the commits should be
merged, and only a minority (sometimes non-existing minority) should
not?

Re: Rewriting bzrmerge.el

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Sat, 15 Nov 2014 18:50:10 +0100
>> 
>> git rebase -i
>
> Wouldn't that unnecessarily annoy, since most of the commits should be
> merged, and only a minority (sometimes non-existing minority) should
> not?

It wouldn't.  You can mark all commits (apart from the first) as s
(squash), and then you get to edit one commit message which is composed
of all the individual commit messages.

Where is the annoyance in that?

But actually, I tend to group a feature branch into several logically
coherent commits.  That's also easy to do with git rebase -i.

-- 
David Kastrup

Re: Rewriting bzrmerge.el

[David Engster]

Eli Zaretskii writes:
>> From: David Kastrup <dak@gnu.org>
>> Date: Sat, 15 Nov 2014 18:50:10 +0100
>> 
>> git rebase -i
>
> Wouldn't that unnecessarily annoy, since most of the commits should be
> merged, and only a minority (sometimes non-existing minority) should
> not?

Since rebase cannot squash merge commits I wouldn't know how it helps in
the first place, but I'm probably being dense.

-David

Re: Rewriting bzrmerge.el

[Stefan Monnier]

> On second thought: Do you mean we should just merge them in a series of
> merge commits (maybe in a temporary branch as to not pollute 'master'
> too much)? That's easy, of course. I was hoping I could somehow
> replicate the workflow we did with Bazaar, where we had one single merge
> commit for each merge.

It doesn't matter if it's a single commit or not.


        Stefan

Re: Rewriting bzrmerge.el

[David Engster]

Stefan Monnier writes:
>> On second thought: Do you mean we should just merge them in a series of
>> merge commits (maybe in a temporary branch as to not pollute 'master'
>> too much)? That's easy, of course. I was hoping I could somehow
>> replicate the workflow we did with Bazaar, where we had one single merge
>> commit for each merge.
>
> It doesn't matter if it's a single commit or not.

OK, I've done the rewrite but I need to clean it up a bit, which I'll do
over the weekend. I've also found that I can actually squash the merges
into one with a little bit of plumbing: simply do a soft reset before
the merges, then do a 'commit-tree' and set the second parent to the
last commit from emacs-24 you have in your last merge. That seems to
work fine since all the commits in the consecutive merges are directly
related, but I'm not really comfortable with it just yet, so I'll keep
that for later...

-David

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Date: Fri, 21 Nov 2014 23:47:44 +0100
> Cc: "Eric S. Raymond" <esr@thyrsus.com>, emacs-devel@gnu.org
> 
> OK, I've done the rewrite but I need to clean it up a bit, which I'll do
> over the weekend.

Thanks.

What are the assumptions about the location of the release branch?  Do
you assume the release branch is in the same cloned repo as master, or
are there provisions for having the branch in another clone?

(See the discussion of the Wiki recommendations for some background
for these questions.)

Re: Rewriting bzrmerge.el

[Stefan Monnier]

> OK, I've done the rewrite but I need to clean it up a bit, which I'll do
> over the weekend.

Cool, thanks.


        Stefan

Re: Rewriting bzrmerge.el

[David Engster]

Eli Zaretskii writes:
> What are the assumptions about the location of the release branch?  Do
> you assume the release branch is in the same cloned repo as master, or
> are there provisions for having the branch in another clone?

As long as you use git-new-workdir, that shouldn't matter. The 'trunk'
checkout should also have a local tracking branch
'emacs-24'. Alternatively, we could default to merge origin/emacs-24,
which wouldn't even require a local tracking branch.

-David

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  esr@thyrsus.com,  emacs-devel@gnu.org
> Date: Sat, 22 Nov 2014 17:13:29 +0100
> 
> Eli Zaretskii writes:
> > What are the assumptions about the location of the release branch?  Do
> > you assume the release branch is in the same cloned repo as master, or
> > are there provisions for having the branch in another clone?
> 
> As long as you use git-new-workdir, that shouldn't matter.

You mean, gitmerge.el will discover that and DTRT by itself?  That's
way cool!

> The 'trunk' checkout should also have a local tracking branch
> 'emacs-24'.

I don't think it's a good idea to rely on that, because it's something
that's too easy to forget.

> Alternatively, we could default to merge origin/emacs-24, which
> wouldn't even require a local tracking branch.

Yes, please, let's do that.

Thanks.

Re: Rewriting bzrmerge.el

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  esr@thyrsus.com,  emacs-devel@gnu.org
>> Date: Sat, 22 Nov 2014 17:13:29 +0100
>
>> 
>> Eli Zaretskii writes:
>> > What are the assumptions about the location of the release branch?  Do
>> > you assume the release branch is in the same cloned repo as master, or
>> > are there provisions for having the branch in another clone?
>> 
>> As long as you use git-new-workdir, that shouldn't matter.
>
> You mean, gitmerge.el will discover that and DTRT by itself?  That's
> way cool!
>
>> The 'trunk' checkout should also have a local tracking branch
>> 'emacs-24'.
>
> I don't think it's a good idea to rely on that, because it's something
> that's too easy to forget.

That's a misunderstanding. The above was not meant as a requirement, but
as a consequence of using git-new-workdir. If you follow
admin/notes/git-workflow and create a separate checkout of the emacs-24
branch, you will automatically have a local tracking branch emacs-24
(also in 'trunk'). At least that's what I am seeing here. I don't know
the inner workings of git-new-workdir, but the directories should to be
the same clone, but with a different checkout. So if you call 'git
branch' in 'trunk', you should see

  emacs-24
* master

and in 'emacs-24'

* emacs-24
  master

That's why gitmerge.el doesn't have to detect anything. It simply can
do its work in 'trunk'.

>> Alternatively, we could default to merge origin/emacs-24, which
>> wouldn't even require a local tracking branch.
>
> Yes, please, let's do that.

You'll be able to do either. For git it makes no difference whether you
merge a local or a remote branch.

-David

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  esr@thyrsus.com,  emacs-devel@gnu.org
> Date: Sat, 22 Nov 2014 18:17:16 +0100
> 
> >> The 'trunk' checkout should also have a local tracking branch
> >> 'emacs-24'.
> >
> > I don't think it's a good idea to rely on that, because it's something
> > that's too easy to forget.
> 
> That's a misunderstanding. The above was not meant as a requirement, but
> as a consequence of using git-new-workdir.

I don't think we should rely on git-new-workdir, either.  First, it
doesn't work well on Windows, so the few frequent contributors
(including yours truly) who work on that system will be unable to use
it.  And second, it's a contrib/ script, i.e. not really official yet,
and has a few caveats (see past discussions).

I'm going to recommend on the Wiki that people who don't consider
themselves advanced/experienced Git users clone the repository twice,
in which case they most probably won't have a checkout of emacs-24 in
the clone where they work on master, and vice versa.

> If you follow admin/notes/git-workflow and create a separate
> checkout of the emacs-24 branch, you will automatically have a local
> tracking branch emacs-24 (also in 'trunk').

Following admin/notes/git-workflow is not a requirement, it's an
advice.  IMO, gitmerge.el should work even without that, at least
ideally.

> That's why gitmerge.el doesn't have to detect anything. It simply can
> do its work in 'trunk'.

AFAIU, not if there's no checkout of emacs-24 there.

> >> Alternatively, we could default to merge origin/emacs-24, which
> >> wouldn't even require a local tracking branch.
> >
> > Yes, please, let's do that.
> 
> You'll be able to do either. For git it makes no difference whether you
> merge a local or a remote branch.

I didn't mean me, I meant gitmerge.el.  If it always uses
origin/emacs-24, it will work regardless of whether there is a
checkout of emacs-24.  So I think this is more reliable.

Re: Rewriting bzrmerge.el

[David Engster]

Eli Zaretskii writes:
> I'm going to recommend on the Wiki that people who don't consider
> themselves advanced/experienced Git users clone the repository twice,
> in which case they most probably won't have a checkout of emacs-24 in
> the clone where they work on master, and vice versa.

If there are problems with git-new-workdir, that's probably for the
best. I've never really used it, especially not on Windows.

>> If you follow admin/notes/git-workflow and create a separate
>> checkout of the emacs-24 branch, you will automatically have a local
>> tracking branch emacs-24 (also in 'trunk').
>
> Following admin/notes/git-workflow is not a requirement, it's an
> advice.  IMO, gitmerge.el should work even without that, at least
> ideally.

It will always be able to merge origin/emacs-24. I guess you could set
things up to merge from a second, separate clone (by adding it as a
remote), but I don't see the need.

>> You'll be able to do either. For git it makes no difference whether you
>> merge a local or a remote branch.
>
> I didn't mean me, I meant gitmerge.el.  If it always uses
> origin/emacs-24, it will work regardless of whether there is a
> checkout of emacs-24.  So I think this is more reliable.

At the moment, gitmerge will ask you what branch to merge with a
completing-read, where you can choose from a list of all local and
remote branches (they can optionally be filtered through a regexp). The
branch 'origin/emacs-24' is now set as INITIAL-INPUT, since this will be
the main purpose of gitmerge (for now). Are you OK with that?

-David

Re: Rewriting bzrmerge.el

[Stefan Monnier]

> AFAIU, not if there's no checkout of emacs-24 there.

"git merge" (and I assume gitmerge.el) doesn't require a checkout of the
branch which you're trying to merge.  It just needs to be in the
repository, which it usually will in the case of merging emacs-24 to
master since they both come from the same repository, so you'd have to
do a "clone" that only fetches "master" in order not to have emacs-24.


        Stefan

Re: Rewriting bzrmerge.el

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I'm going to recommend on the Wiki that people who don't consider
> themselves advanced/experienced Git users clone the repository twice,
> in which case they most probably won't have a checkout of emacs-24 in
> the clone where they work on master, and vice versa.

From previous discussion on this list, it seems like most people don't
want to download the repository twice (or store two copies of it on
disk), so they want a different workflow.

It would be nice it it were possible to provide them with that.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Rewriting bzrmerge.el

[Stephen J. Turnbull]

Lars Magne Ingebrigtsen writes:

 >From previous discussion on this list, it seems like most people don't
 > want to download the repository twice (or store two copies of it on
 > disk), so they want a different workflow.

What you're missing is that the people to whom that description
applies are mostly (a) experienced git users (who have their own
workflows for git and don't need the workflow document to tell them
how to suck eggs) or (b) "drive-by" contributors (who won't be doing
much if any merging of stable to trunk, the primary use case for the
multiple clone model).

 > It would be nice it it were possible to provide them with that.

But providing choices and trying to give guidance as to which ones are
efficient in the presence of conflicting definitions of "efficient"
(nothing that involves frequent bootstrap is time-efficient!) adds
complexity, a disservice to the intended audience (eg, Richard and
Ken'ichi).

Steve

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  esr@thyrsus.com,  emacs-devel@gnu.org
> Date: Sat, 22 Nov 2014 22:11:18 +0100
> 
> At the moment, gitmerge will ask you what branch to merge with a
> completing-read, where you can choose from a list of all local and
> remote branches (they can optionally be filtered through a regexp). The
> branch 'origin/emacs-24' is now set as INITIAL-INPUT, since this will be
> the main purpose of gitmerge (for now). Are you OK with that?

Yes, thanks.

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: David Engster <deng@randomsample.de>,  esr@thyrsus.com,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sun, 23 Nov 2014 00:02:18 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I'm going to recommend on the Wiki that people who don't consider
> > themselves advanced/experienced Git users clone the repository twice,
> > in which case they most probably won't have a checkout of emacs-24 in
> > the clone where they work on master, and vice versa.
> 
> >From previous discussion on this list, it seems like most people don't
> want to download the repository twice (or store two copies of it on
> disk), so they want a different workflow.
> 
> It would be nice it it were possible to provide them with that.

There's any number of ways to handle this problem with Git.  We cannot
possibly describe all of them on the Wiki, not without confusing the
heck out of the reader.  Experienced Git users will be able to find
the way that suits them even without reading the Wiki.

In any case, I will mention git-new-workdir, which seems to be the
other popular method.  And there's always a possibility of changing
the recommendations in the future, if experience shows that changes
are needed.

Re: Rewriting bzrmerge.el

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Lars Magne Ingebrigtsen writes:
>
>  >From previous discussion on this list, it seems like most people don't
>  > want to download the repository twice (or store two copies of it on
>  > disk), so they want a different workflow.

"Download the repository twice" seems uncalled for.  One can always use

git clone --mirror

to create the second clone.

-- 
David Kastrup

Re: Rewriting bzrmerge.el

[Lars Magne Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

>  > It would be nice it it were possible to provide them with that.
>
> But providing choices and trying to give guidance as to which ones are
> efficient in the presence of conflicting definitions of "efficient"
> (nothing that involves frequent bootstrap is time-efficient!) adds
> complexity, a disservice to the intended audience (eg, Richard and
> Ken'ichi).

I don't want to provide choices and complexity.  I want to provide
simple instructions.  But the better the results ends up being, the
better the instructions are.

If there is a way to get two separate branches checked out at the same
time without doubling the disk spaces used, that would be nice.  The
answer currently seems to be "use git-new-workdir", unless there's
something one of the git experts has said somewhere that I missed, which
is entirely possible.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Rewriting bzrmerge.el

[Ken Brown]

On 11/23/2014 9:22 AM, Lars Magne Ingebrigtsen wrote:
> If there is a way to get two separate branches checked out at the same
> time without doubling the disk spaces used, that would be nice.  The
> answer currently seems to be "use git-new-workdir", unless there's
> something one of the git experts has said somewhere that I missed, which
> is entirely possible.

http://lists.gnu.org/archive/html/emacs-devel/2014-11/msg00821.html

Re: Rewriting bzrmerge.el

[Stephen J. Turnbull]

Lars Magne Ingebrigtsen writes:

 > If there is a way to get two separate branches checked out at the same
 > time without doubling the disk spaces used, that would be nice.

How much more than "negligible" is unacceptable, and how long before
you cross that line is acceptable?

You see, git is not bzr or hg (yes, I know about bzr shared repos, but
my bzr setup with about 5 Emacs branches including trunk, emacs-24,
and emacs-23 uses about 3X as much space as my git repo with all Emacs
git branches in it).  Simply doing "git clone source-repo target-repo"
uses very little extra space (or time) at clone time because the
objects are hard-linked across directories by default.  Of course this
will tend to diverge over time because new commits often won't be
shared, and granularity may be a few MB due to packed objects.  Still,
given the amount of existing history in an Emacs repo
-- most of which will never be GC'd -- I'd be surprised if the extra
space took less than 5 years to reach 150MB (~50% of current Emacs
repos), and I wouldn't be surprised if it took 10 years.  (No, I don't
have any actual data.)

Also, the primary use case Eli talks about is doing synchs of stable
to trunk, and the occasional cherrypick in the other direction.  In
this case, the "trunk" repository will have a copy of the stable
branch, and in the unlikely case they diverge much, you can always
"repack" the "stable" repo by fetching stable-repo into trunk-repo,
then rm -rf and git clone a new stable-repo.  Most of the time is
taken up by rm -rf ....  Whether that hack works for your favorite
feature branches, I don't know (you might not want to pollute trunk
with them before they're ready), but it probably does.

I don't see any good reason for describing any other strategies than
"if you're worried about bootstrap times, just use a second clone from
your local trunk repo".  If somebody really runs into a space issue,
it's easy enough to explain how to reduce space usage.

 > The answer currently seems to be "use git-new-workdir"

This should work OK 99.44% of the time in Emacs workflows, but if the
new user gets frisky with rebasing or commit --amend etc, they *can*
shoot themselves in the foot.  If they do, they almost certainly will
break a private part of the repo that cannot be recovered from public
sources.  I don't think that's worth it, IMHO FWIW YMMV.

Re: Rewriting bzrmerge.el

[Lars Magne Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Simply doing "git clone source-repo target-repo"
> uses very little extra space (or time) at clone time because the
> objects are hard-linked across directories by default. 

Well, then that seems like the solution we should recommend.  In that
case, I don't really understand what the point of git-new-workdir is?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Sun, 23 Nov 2014 09:01:25 +0100
> 
> "Download the repository twice" seems uncalled for.  One can always use
> 
> git clone --mirror
> 
> to create the second clone.

But I thought the discussion demonstrated that this command alone does
not guarantee the second clone will have all the branches.  Did I
misunderstand?

Re: Rewriting bzrmerge.el

[Achim Gratz]

Eli Zaretskii writes:
>> "Download the repository twice" seems uncalled for.  One can always use
>> 
>> git clone --mirror
>> 
>> to create the second clone.
>
> But I thought the discussion demonstrated that this command alone does
> not guarantee the second clone will have all the branches.  Did I
> misunderstand?

It does...  But it creates a bare repository, which you then have to
move into the "emacs-24" directory as .git/ or provide a gitlink file
that points to it.

Again, the easiest way to clone the repo twice is to do exactly that and
have the second clone use the first as a reference repository.  A more
symmetric setup is to do the first clone into a bare repository and then
clone both working repos with reference to the first one:

git clone --bare git://git.sv.gnu.org/emacs.git
git clone --reference emacs.git --branch master   master
git clone --reference emacs.git --branch emacs-24 emacs-24

If you remember to always do a "git remote update" first for the bare
repository before pulling in the work trees, you'll never copy any
object from the official repo more than once.  This also scales nicely
to even more repositories as long as you really keep a strict "one repo
per branch" workflow.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Rewriting bzrmerge.el

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Sun, 23 Nov 2014 09:01:25 +0100
>> 
>> "Download the repository twice" seems uncalled for.  One can always use
>> 
>> git clone --mirror
>> 
>> to create the second clone.
>
> But I thought the discussion demonstrated that this command alone does
> not guarantee the second clone will have all the branches.  Did I
> misunderstand?

I thought that the main problem was that you needed to reconfig the
repository as non-bare afterwards.

-- 
David Kastrup

Re: Rewriting bzrmerge.el

[Steinar Bang]

>>>>> David Engster <deng@randomsample.de>:

> It will always be able to merge origin/emacs-24. I guess you could set
> things up to merge from a second, separate clone (by adding it as a
> remote), but I don't see the need.

You don't need a clone for this, just a local bare repo.

I'm assuming that the two separate clones are %USERPROFILE%\emacs\master
and %USERPROFILE%\emacs\master for the examples above.

Open a magit bash window, and do this:
 mkdir -p $HOME/repositories/local/
 (cd $HOME/repositories/local; rm -rf emacs.git; git init --bare emacs.git)
 cd $HOME/emacs/master
 git remote add local $HOME/repositories/local/emacs.git
 cd $HOME/emacs/emacs-24
 git remote add local $HOME/repositories/local/emacs.git

And now you're ready to go.

If you have added a change to emacs-24 and want it on master, then do
this (in magit bash):
 cd $HOME/emacs/emacs-24
 git push local HEAD
 cd $HOME/emacs/master
 git fetch local
 git merge local/emacs-24

(if you were using a feature branch instead of emacs-24, you could just
push it instead of "emacs-24" that was used in the example).

To speed things up and keep the local repo small, you can do this:
 (cd $HOME/repositories/outgoing/emacs.git; git gc --prune=now --aggressive)

If you have lots of clutter in the local repo (eg. lots of defunct
feature branches), you can just do this and start fresh:
 (cd $HOME/repositories/local; rm -rf emacs.git; git init --bare emacs.git)

One reason for using magit bash instead of using git from regular
cmd.exe (which I also sometimes do), is that magit bash does TAB expand
on tags and branch names.

Re: Rewriting bzrmerge.el

[Steinar Bang]

>>>>> David Kastrup <dak@gnu.org>:

> "Download the repository twice" seems uncalled for.  One can always use

> git clone --mirror

> to create the second clone.

Or just copy the first clone... command examples from magit bash:
 mkdir -p $HOME/emacs
 cd $HOME/emacs
 git clone username@git.sv.gnu.org:/srv/git/emacs.git master
 cp -a master emacs-24
 cd $HOME/emacs/emacs-24
 git checkout emacs-24

Re: Rewriting bzrmerge.el

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: David Kastrup <dak@gnu.org>
 > > Date: Sun, 23 Nov 2014 09:01:25 +0100
 > > 
 > > "Download the repository twice" seems uncalled for.  One can always use
 > > 
 > > git clone --mirror
 > > 
 > > to create the second clone.
 > 
 > But I thought the discussion demonstrated that this command alone does
 > not guarantee the second clone will have all the branches.  Did I
 > misunderstand?

--mirror does mean it has all branches that are present in the first
clone (including all tracking branches).  What it doesn't have is a
checked-out workspace (it's a bare repository).

Re: Rewriting bzrmerge.el

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Eli Zaretskii writes:
>  > > From: David Kastrup <dak@gnu.org>
>  > > Date: Sun, 23 Nov 2014 09:01:25 +0100
>  > > 
>  > > "Download the repository twice" seems uncalled for.  One can always use
>  > > 
>  > > git clone --mirror
>  > > 
>  > > to create the second clone.
>  > 
>  > But I thought the discussion demonstrated that this command alone does
>  > not guarantee the second clone will have all the branches.  Did I
>  > misunderstand?
>
> --mirror does mean it has all branches that are present in the first
> clone (including all tracking branches).  What it doesn't have is a
> checked-out workspace (it's a bare repository).

dak@lola:/usr/local/tmp$ git clone --mirror --no-bare lilypond lilypond2/.git
Cloning into bare repository 'lilypond2/.git'...
done.
dak@lola:/usr/local/tmp$ cd lilypond2/
dak@lola:/usr/local/tmp/lilypond2$ ls
dak@lola:/usr/local/tmp/lilypond2$ git config core.bare false
dak@lola:/usr/local/tmp/lilypond2$ git branch
  badmike
  bettertie
  bookactive
  chordrootnamer
  clean-scm
[...]

but

dak@lola:/usr/local/tmp/lilypond2$ cat .git/config
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
[remote "origin"]
	url = /usr/local/tmp/lilypond
	fetch = +refs/*:refs/*
	mirror = true

and I am not really sure what the implications of that are.  It's
definitely not an equivalent copy...

-- 
David Kastrup

Re: Rewriting bzrmerge.el

[David Engster]

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

David Engster writes:
> Stefan Monnier writes:
>>> On second thought: Do you mean we should just merge them in a series of
>>> merge commits (maybe in a temporary branch as to not pollute 'master'
>>> too much)? That's easy, of course. I was hoping I could somehow
>>> replicate the workflow we did with Bazaar, where we had one single merge
>>> commit for each merge.
>>
>> It doesn't matter if it's a single commit or not.
>
> OK, I've done the rewrite but I need to clean it up a bit, which I'll do
> over the weekend.

First iteration is attached. See comment for a quick intro. Please tell
me if it's OK to commit or if larger changes are needed.

If anyone starts hacking on it, please try to keep it general, meaning:
do not hard-code branches or repositories. My plan is to extend this for
the CEDET merge (in fact, the whole major mode thing is mostly lifted
from my existing CEDET merge script).

There are bugs, obviously, so don't push too fast. Also, you'll see that
our regexp for detecting backports is pretty loose...

Also note that - as I've already written - skipped commits will create a
separate merge commit (it will combine consecutive skipped commits,
though). Squashing those merge commits can be done, but I've not yet
built that in.

-David


[-- Attachment #2: gitmerge.el --]
[-- Type: application/emacs-lisp, Size: 19850 bytes --]

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: David Kastrup <dak@gnu.org>,
>     emacs-devel@gnu.org
> Date: Mon, 24 Nov 2014 01:42:39 +0900
> 
> --mirror does mean it has all branches that are present in the first
> clone (including all tracking branches).  What it doesn't have is a
> checked-out workspace (it's a bare repository).

Sorry, I don't understand what that means, and what are the
implications on the user of such a clone.  Can you elaborate, please?

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 23 Nov 2014 17:29:38 +0100
> 
> Again, the easiest way to clone the repo twice is to do exactly that and
> have the second clone use the first as a reference repository.  A more
> symmetric setup is to do the first clone into a bare repository and then
> clone both working repos with reference to the first one:
> 
> git clone --bare git://git.sv.gnu.org/emacs.git
> git clone --reference emacs.git --branch master   master
> git clone --reference emacs.git --branch emacs-24 emacs-24
> 
> If you remember to always do a "git remote update" first for the bare
> repository before pulling in the work trees, you'll never copy any
> object from the official repo more than once.  This also scales nicely
> to even more repositories as long as you really keep a strict "one repo
> per branch" workflow.

Thanks, but how is this better than just 2 full clones of the upstream
repo on Savannah?  What you suggest is certainly more complicated,
albeit slightly so.  What do we gain except disk savings?  Scalability
is not really an issue, as it's hard for me to imagine that Emacs will
have more than 2 divergent branches any time soon.

Re: Rewriting bzrmerge.el

[David Engster]

Steinar Bang writes:
>>>>>> David Engster <deng@randomsample.de>:
>
>> It will always be able to merge origin/emacs-24. I guess you could set
>> things up to merge from a second, separate clone (by adding it as a
>> remote), but I don't see the need.
>
> You don't need a clone for this, just a local bare repo.

[...]

I know, but you have snipped the context. The problem was: what's the
most robust solution for people who want to have separate checkouts of
branches, but which is also (a) easy to setup, (b) works well on
Windows, and (c) is manageable for people without much git
experience. And the answer for this seems to be: create two separate
clones, and just live with the additional HD space this takes up. Then
Eli's question was whether gitmerge.el could merge from such a separate
clone, to which my answer was: I guess you could, but there's no need,
let's just merge the remote branch by default.

-David

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Sun, 23 Nov 2014 17:32:57 +0100
> 
> >>>>> David Engster <deng@randomsample.de>:
> 
> > It will always be able to merge origin/emacs-24. I guess you could set
> > things up to merge from a second, separate clone (by adding it as a
> > remote), but I don't see the need.
> 
> You don't need a clone for this, just a local bare repo.

The clone is needed to actually work on the release branch, without
having to bootstrap it each time you "git checkout" it.

Re: Rewriting bzrmerge.el

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

>> >>>>> David Engster <deng@randomsample.de>:

>> > It will always be able to merge origin/emacs-24. I guess you could set
>> > things up to merge from a second, separate clone (by adding it as a
>> > remote), but I don't see the need.

>> You don't need a clone for this, just a local bare repo.

> The clone is needed to actually work on the release branch, without
> having to bootstrap it each time you "git checkout" it.

I know.

The bare repo referred above to is for transferring a branch between two
separate clones on local disk.  The bare repo is separate from the two
clones named emacs24 and master.

The snipped procedure shows how to create that local repository, and how
to push a branch from one clone and merge it into a branch in the other
clone.

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Sun, 23 Nov 2014 19:40:40 +0100
> 
> > The clone is needed to actually work on the release branch, without
> > having to bootstrap it each time you "git checkout" it.
> 
> I know.
> 
> The bare repo referred above to is for transferring a branch between two
> separate clones on local disk.

What for?  Just "git pull" in the other clone will do that, easier and
with fewer complications.

> The snipped procedure shows how to create that local repository, and how
> to push a branch from one clone and merge it into a branch in the other
> clone.

I don't think it's a good idea to recommend such procedures for Git
newbies.  The potential for mistakes is too high, IMO.

Re: Rewriting bzrmerge.el

[Stefan Monnier]

Could you guys move this to some other thread.  gitmerge.el is unrelated
to the issue of multiple checkout or not, since it works on branches not
on checkouts.


        Stefan


>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> Eli Zaretskii <eliz@gnu.org> writes:
>> I'm going to recommend on the Wiki that people who don't consider
>> themselves advanced/experienced Git users clone the repository twice,
>> in which case they most probably won't have a checkout of emacs-24 in
>> the clone where they work on master, and vice versa.

>> From previous discussion on this list, it seems like most people don't
> want to download the repository twice (or store two copies of it on
> disk), so they want a different workflow.

> It would be nice it it were possible to provide them with that.

> -- 
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Eli Zaretskii <eliz@gnu.org>, David Engster <deng@randomsample.de>,
>         esr@thyrsus.com, emacs-devel@gnu.org
> Date: Sun, 23 Nov 2014 22:59:03 -0500
> 
> Could you guys move this to some other thread.  gitmerge.el is unrelated
> to the issue of multiple checkout or not, since it works on branches not
> on checkouts.

It was not clear it's unrelated when this started.  Besides, I don't
see why it is so important in which thread this issue is discussed, so
much so that you felt the need to reprimand us.

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Mon, 24 Nov 2014 00:11:12 +0900
> Cc: emacs-devel@gnu.org
> 
> yes, I know about bzr shared repos, but
> my bzr setup with about 5 Emacs branches including trunk, emacs-24,
> and emacs-23 uses about 3X as much space as my git repo with all Emacs
> git branches in it

That's not really a fair comparison, as it includes the work trees, of
which the Git repo has only one.

It would be more fair to compare the .bzr subdirectory of the Bazaar
repository with the .git subdirectory of the Git repository, in which
case Git wins with a 1:2 ratio (which might deteriorate to some extent
in the future, since the initial repository was very aggressively
packed).

> Simply doing "git clone source-repo target-repo" uses very little
> extra space (or time) at clone time because the objects are
> hard-linked across directories by default.

That is true (even on Windows), but then one would need to pull twice,
once from upstream, the other time from the first clone to the second,
right?  Doesn't sound a net win to me.

> I don't see any good reason for describing any other strategies than
> "if you're worried about bootstrap times, just use a second clone from
> your local trunk repo".

That's what I will probably write, yes.

>  > The answer currently seems to be "use git-new-workdir"
> 
> This should work OK 99.44% of the time in Emacs workflows, but if the
> new user gets frisky with rebasing or commit --amend etc, they *can*
> shoot themselves in the foot.  If they do, they almost certainly will
> break a private part of the repo that cannot be recovered from public
> sources.  I don't think that's worth it, IMHO FWIW YMMV.

I will probably mention git-new-workdir, but not more than that.
Interested readers can read the documentation and set that up.

Thanks.

Re: Rewriting bzrmerge.el

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > It would be more fair to compare the .bzr subdirectory of the Bazaar
 > repository with the .git subdirectory of the Git repository, in which
 > case Git wins with a 1:2 ratio

That's what I mean by "repo", and that's what I did.  Looking more
closely, I see some obsolete-packs which account for 1/3 of the space.
However, I've not made any commits to those branches, so they came
with the original clone somehow.

 > > Simply doing "git clone source-repo target-repo" uses very little
 > > extra space (or time) at clone time because the objects are
 > > hard-linked across directories by default.
 > 
 > That is true (even on Windows), but then one would need to pull twice,
 > once from upstream, the other time from the first clone to the second,
 > right?  Doesn't sound a net win to me.

Fetching takes negligible time locally, and if you're worried about
the typing write a shell function/script or an Emacs command.  OTOH,
if the merge part of pull breaks, it would break anyway, and you have
to fix that.

I think there's a huge amount of premature optimization going on here
(not from you: "make bootstrap" is an unacceptable time sink if you
can avoid it, finding ways to avoid it is not "premature").  Both the
space worries of people less experienced with git and the workflow
proposals of the experienced users really should be postponed until
the new git users actually run into those issues.

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: larsi@gnus.org,
>     emacs-devel@gnu.org
> Date: Tue, 25 Nov 2014 03:18:53 +0900
> 
>  > > Simply doing "git clone source-repo target-repo" uses very little
>  > > extra space (or time) at clone time because the objects are
>  > > hard-linked across directories by default.
>  > 
>  > That is true (even on Windows), but then one would need to pull twice,
>  > once from upstream, the other time from the first clone to the second,
>  > right?  Doesn't sound a net win to me.
> 
> Fetching takes negligible time locally, and if you're worried about
> the typing write a shell function/script or an Emacs command.

I'm worried about people forgetting the 2nd fetch, and then getting
themselves into trouble.

> I think there's a huge amount of premature optimization going on here
> (not from you: "make bootstrap" is an unacceptable time sink if you
> can avoid it, finding ways to avoid it is not "premature").  Both the
> space worries of people less experienced with git and the workflow
> proposals of the experienced users really should be postponed until
> the new git users actually run into those issues.

I agree wholeheartedly.

Re: Rewriting bzrmerge.el

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > >  > That is true (even on Windows), but then one would need to pull twice,
 > >  > once from upstream, the other time from the first clone to the second,
 > >  > right?  Doesn't sound a net win to me.
 > > 
 > > Fetching takes negligible time locally, and if you're worried about
 > > the typing write a shell function/script or an Emacs command.
 > 
 > I'm worried about people forgetting the 2nd fetch, and then getting
 > themselves into trouble.

Sure, but how do you avoid a second fetch or pull (this is what
updates the branch head) if you have a 2d clone?  I'm assuming that is
the way to go for some workflows, specifically if you're working on
the release branch and the trunk a lot.  Otherwise, I'm all in favor
of a single repo with the branches of interest being colocated.

Re: Rewriting bzrmerge.el

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: larsi@gnus.org,
>     emacs-devel@gnu.org
> Date: Tue, 25 Nov 2014 03:59:08 +0900
> 
> Eli Zaretskii writes:
> 
>  > >  > That is true (even on Windows), but then one would need to pull twice,
>  > >  > once from upstream, the other time from the first clone to the second,
>  > >  > right?  Doesn't sound a net win to me.
>  > > 
>  > > Fetching takes negligible time locally, and if you're worried about
>  > > the typing write a shell function/script or an Emacs command.
>  > 
>  > I'm worried about people forgetting the 2nd fetch, and then getting
>  > themselves into trouble.
> 
> Sure, but how do you avoid a second fetch or pull (this is what
> updates the branch head) if you have a 2d clone?

Well, then I was talking about the 3rd fetch.