[PATCH] nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}

[PATCH] nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

With two branches getting fetched (master and config), the branch
referenced by FETCH_HEAD is ambiguous.  For example, I have:

  $ cat FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0 \
    not-for-merge   branch 'config' of http://nmbug.tethera.net/git/nmbug-tags
  acd379ccb973c45713eee9db177efc530f921954 \
    not-for-merge   branch 'master' of http://nmbug.tethera.net/git/nmbug-tags

(where I wrapped the line by hand).  This means that FETCH_HEAD
references the config branch:

  $ git rev-parse FETCH_HEAD
  41d7bfa7184cc93c9dac139d1674e9530799e3b0

which breaks all of the FETCH_HEAD logic in nmbug (where FETCH_HEAD is
assumed to point to the master branch).

Instead of relying on FETCH_HEAD, use @{upstream} as the
remote-tracking branch that should be merged/diffed/integrated into
HEAD.  @{upstream} was added in Git v1.7.0 (2010-02-12) [1], so
relying on it should be fairly safe.  One tricky bit is that bare
repositories don't set upstream tracking branches by default:

  $ git clone --bare http://nmbug.tethera.net/git/nmbug-tags.git nmbug-bare
  $ cd nmbug-bare
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Local refs configured for 'git push':
      config pushes to config (up to date)
      master pushes to master (up to date)

While in a non-bare clone:

  $ git clone http://nmbug.tethera.net/git/nmbug-tags.git
  $ cd nmbug-tags
  $ git remote show origin
  * remote origin
    Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
    Push  URL: http://nmbug.tethera.net/git/nmbug-tags.git
    HEAD branch: master
    Remote branches:
      config tracked
      master tracked
    Local branch configured for 'git pull':
      master merges with remote master
    Local ref configured for 'git push':
      master pushes to master (up to date)

From the clone docs [2]:

  --bare::
        Make a 'bare' Git repository…
        Also the branch heads at the remote are copied directly
        to corresponding local branch heads, without mapping
        them to `refs/remotes/origin/`.  When this option is
        used, neither remote-tracking branches nor the related
        configuration variables are created.

To use @{upstream}, we need to restore the local vs. remote-tracking
distinction, so this commit adds 'nmbug clone', which adds the
following configuration adjustments:

  $ git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'
  $ git config branch.$branch.remote origin
  $ git config branch.$branch.merge refs/heads/$branch

where $branch is the branch referenced by the remote's HEAD, which
'git clone' uses to setup the local branch.  After tweaking the
config, we populate the new remote-tracking branches with:

  $ git fetch origin

After which:

  $ git rev-parse @{upstream}
  acd379ccb973c45713eee9db177efc530f921954

gives us the master-branch commit.  Existing nmbug users will have to
run the configuration tweaks and re-fetch by hand.  If you don't have
any local commits, you could also blow away your NMBGIT repository and
re-clone from scratch:

  $ nmbug clone http://nmbug.tethera.net/git/nmbug-tags.git

Besides removing the ambiguity of FETCH_HEAD, this commit allows users
to configure which upstream branch they want nmbug to track via 'git
config', in case they want to change their upstream repository.

[1]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/RelNotes/1.7.0.txt
[2]: http://git.kernel.org/cgit/git/git.git/tree/Documentation/git-clone.txt
---
 devel/nmbug/nmbug | 49 +++++++++++++++++++++++++++++++++++--------------
 1 file changed, 35 insertions(+), 14 deletions(-)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 90d98b6..3c1d597 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -26,6 +26,7 @@ my $ESCAPED_RX =	qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
 my %command = (
 	     archive	=> \&do_archive,
 	     checkout	=> \&do_checkout,
+	     clone	=> \&do_clone,
 	     commit	=> \&do_commit,
 	     fetch	=> \&do_fetch,
 	     help	=> \&do_help,
@@ -125,6 +126,17 @@ sub do_archive {
   system ('git', "--git-dir=$NMBGIT", 'archive', 'HEAD');
 }
 
+sub do_clone {
+  my $repository = shift;
+  system ('git', 'clone', '--bare', $repository, $NMBGIT) == 0
+    or die "'git clone' exited with nonzero value\n";
+  my $branch = git ('symbolic-ref', 'HEAD');
+  $branch =~ s|^refs/heads/||;
+  git ('config', 'remote.origin.fetch', '+refs/heads/*:refs/remotes/origin/*');
+  git ('config', "branch.$branch.remote", 'origin');
+  git ('config', "branch.$branch.merge", 'refs/heads/$branch');
+  do_fetch();
+}
 
 sub is_committed {
   my $status = shift;
@@ -332,21 +344,24 @@ To discard your changes,  run 'nmbug checkout'
 
 sub do_pull {
   my $remote = shift || 'origin';
+  my $branch = shift || 'master';
 
   git ( 'fetch', $remote);
 
-  do_merge ();
+  do_merge ("$remote/$branch");
 }
 
 
 sub do_merge {
+  my $commit = shift || '@{upstream}';
+
   insist_committed ();
 
   my $tempwork = tempdir ('/tmp/nmbug-merge.XXXXXX', CLEANUP => 1);
 
   git ( { GIT_WORK_TREE => $tempwork }, 'checkout', '-f', 'HEAD');
 
-  git ( { GIT_WORK_TREE => $tempwork }, 'merge', 'FETCH_HEAD');
+  git ( { GIT_WORK_TREE => $tempwork }, 'merge', $commit);
 
   do_checkout ();
 }
@@ -407,13 +422,12 @@ sub do_status {
 
 
 sub is_unmerged {
+  my $commit = shift || '@{upstream}';
 
-  return 0 if (! -f $NMBGIT.'/FETCH_HEAD');
-
-  my $fetch_head = git ('rev-parse', 'FETCH_HEAD');
-  my $base = git ( 'merge-base', 'HEAD', 'FETCH_HEAD');
+  my $upstream = git ('rev-parse', $commit);
+  my $base = git ( 'merge-base', 'HEAD', $commit);
 
-  return ($base ne $fetch_head);
+  return ($base ne $upstream);
 
 }
 
@@ -473,7 +487,7 @@ sub diff_index {
 sub diff_refs {
   my $filter = shift;
   my $ref1 = shift || 'HEAD';
-  my $ref2 = shift || 'FETCH_HEAD';
+  my $ref2 = shift || '@{upstream}';
 
   my $fh= git_pipe ( 'diff', "--diff-filter=$filter", '--name-only',
 		 $ref1, $ref2);
@@ -561,10 +575,11 @@ git. Any extra arguments are used (one per line) as a commit message.
 
 push local nmbug git state to remote repo
 
-=item  B<pull> [remote]
+=item  B<pull> [remote] [branch]
 
 pull (merge) remote repo changes to notmuch. B<pull> is equivalent to
-B<fetch> followed by B<merge>.
+B<fetch> followed by B<merge>.  The default remote is C<origin>, and
+the default branch is C<master>.
 
 =back
 
@@ -572,6 +587,12 @@ B<fetch> followed by B<merge>.
 
 =over 8
 
+=item B<clone> repository
+
+Create a local nmbug repository from a remote source.  This wraps
+C<git clone --bare>, adding some configuration adjustments to setup
+remote-tracking branches and upstreams.
+
 =item B<checkout>
 
 Update the notmuch database from git. This is mainly useful to discard
@@ -589,12 +610,12 @@ print help [for subcommand]
 =item B<log> [parameters]
 
 A simple wrapper for git log. After running C<nmbug fetch>, you can
-inspect the changes with C<nmbug log HEAD..FETCH_HEAD>
+inspect the changes with C<nmbug log HEAD..@{upstream}>
 
-=item B<merge>
+=item B<merge> [commit]
 
-Merge changes from FETCH_HEAD into HEAD, and load the result into
-notmuch.
+Merge changes from C<commit> into HEAD, and load the result into
+notmuch.  The default commit is C<@{upstream}>.
 
 =item  B<status>
 
-- 
1.8.3.2

Re: [PATCH] nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:


> +sub do_clone {
> +  my $repository = shift;
> +  system ('git', 'clone', '--bare', $repository, $NMBGIT) == 0
> +    or die "'git clone' exited with nonzero value\n";
> +  my $branch = git ('symbolic-ref', 'HEAD');
> +  $branch =~ s|^refs/heads/||;
> +  git ('config', 'remote.origin.fetch', '+refs/heads/*:refs/remotes/origin/*');
> +  git ('config', "branch.$branch.remote", 'origin');
> +  git ('config', "branch.$branch.merge", 'refs/heads/$branch');
> +  do_fetch();
> +}

I think doing a fetch immediately after a clone deserves a comment.

>  
> @@ -473,7 +487,7 @@ sub diff_index {
>  sub diff_refs {
>    my $filter = shift;
>    my $ref1 = shift || 'HEAD';
> -  my $ref2 = shift || 'FETCH_HEAD';
> +  my $ref2 = shift || '@{upstream}';

I wonder about hard-coding '@{upstream}' in several places. What do you
think about a global "constant"? OF course it was hard-coded FETCH_HEAD
before, but I think not in quite as many places.

Re: [PATCH] nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3133 bytes --]

On Sat, Mar 08, 2014 at 08:43:26AM -0400, David Bremner wrote:
> W. Trevor King writes:
> > +sub do_clone {
> > +  my $repository = shift;
> > +  system ('git', 'clone', '--bare', $repository, $NMBGIT) == 0
> > +    or die "'git clone' exited with nonzero value\n";
> > +  my $branch = git ('symbolic-ref', 'HEAD');
> > +  $branch =~ s|^refs/heads/||;
> > +  git ('config', 'remote.origin.fetch', '+refs/heads/*:refs/remotes/origin/*');
> > +  git ('config', "branch.$branch.remote", 'origin');
> > +  git ('config', "branch.$branch.merge", 'refs/heads/$branch');
> > +  do_fetch();

Oops, it looks like I used a tab instead of two spaces to indent
do_fetch.  I'll fix that in v2.

> > +}
> 
> I think doing a fetch immediately after a clone deserves a comment.

I commented in the commit message.  Basically everything from “One
tricky bit is that bare repositories don't set upstream tracking
branches by default” down through “gives us the master-branch commit.”
is describing what's going on here, and why I made these changes.

On the other hand, perhaps this is too many hoops to jump through to
get upstreams without a working directory.  As an alternative
approach, we could drop --bare and use --no-checkout on the clone.
That would also let us avoid the bare-recovery config adjustments and
follow-up fetch.  With do_clone reduced to just wrapping the 'git
clone' command, we could remove it and just recommend folks run:

  $ git clone --no-checkout http://nmbug.tethera.net/git/nmbug-tags.git ~/.nmbug

That's going to create ~/.nmbug/.git though, if we want to preserve
the current bare-style ~/.nmbug/ layout, we'd need something like:

  $ TEMPDIR=$(mktemp -d)
  $ git clone --no-checkout --separate-git-dir=~/.nmbug \
  >   http://nmbug.tethera.net/git/nmbug-tags.git "${TEMPDIR}/nmbug"
  $ rm -rf "${TEMPDIR}"

If we go the --separate-git-dir route, I think we're better off hiding
that in Perl ;).

Of the three approaches:

1. git clone --bare …, tweak configs, and git fetch
2. git clone --nocheckout …
3. git clone --nocheckout --separate-git-dir …

I prefer the third, but I'm happy with any of them.

> > @@ -473,7 +487,7 @@ sub diff_index {
> >  sub diff_refs {
> >    my $filter = shift;
> >    my $ref1 = shift || 'HEAD';
> > -  my $ref2 = shift || 'FETCH_HEAD';
> > +  my $ref2 = shift || '@{upstream}';
> 
> I wonder about hard-coding '@{upstream}' in several places. What do
> you think about a global "constant"? OF course it was hard-coded
> FETCH_HEAD before, but I think not in quite as many places.

In the three non-comment places where I've hard-coded it, it's just a
default in the case that the caller left that argument empty.  In the
two comment places where I've hard-coded it, one is just mentioning
that it is the default, and the other is giving it as a useful
example.  I don't think that calls for another layer of indirection.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

On Sat, Mar 08, 2014 at 08:26:33AM -0800, W. Trevor King wrote:
> That's going to create ~/.nmbug/.git though, if we want to preserve
> the current bare-style ~/.nmbug/ layout, we'd need something like:
> 
>   $ TEMPDIR=$(mktemp -d)
>   $ git clone --no-checkout --separate-git-dir=~/.nmbug \
>   >   http://nmbug.tethera.net/git/nmbug-tags.git "${TEMPDIR}/nmbug"
>   $ rm -rf "${TEMPDIR}"

Ah, we'd also want to remove the worktree link to the ephemeral
"${TEMPDIR}" with:

  $ git config --unset core.worktree

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] nmbug: Add 'clone' and replace FETCH_HEAD with @{upstream}

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:
>> 
>> I think doing a fetch immediately after a clone deserves a comment.
>
> I commented in the commit message.  Basically everything from “One
> tricky bit is that bare repositories don't set upstream tracking
> branches by default” down through “gives us the master-branch commit.”
> is describing what's going on here, and why I made these changes.

Yes.  To expand my comment, I think some of that information
should be in the code.

I'm not really sure about about the alternatives you mention.