Multiple checkout copies

Multiple checkout copies

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Using git, is there a way to check out copies of different branches in
parallel (in different directories) from the same clone of the
repository?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Multiple checkout copies

[Paul Eggert]

Richard Stallman wrote:
> Using git, is there a way to check out copies of different branches in
> parallel (in different directories) from the same clone of the
> repository?

A popular way to do this is the shell command git-new-workdir, commonly located 
at /usr/share/doc/git/contrib/workdir/git-new-workdir.

I don't use it myself, as 'git clone' is plenty fast and avoids some problems 
when multiple branches trying to update the same repository.  I suspect this is 
why git-new-workdir isn't in PATH by default.

Re: Multiple checkout copies

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I don't use it myself, as 'git clone' is plenty fast and avoids some problems 
  > when multiple branches trying to update the same repository.

Doesn't 'git clone' download a copy of the whole repository?
I thought that is what it did.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

Re: Multiple checkout copies

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 432 bytes --]

On 2 February 2015 at 21:35, Richard Stallman <rms@gnu.org> wrote:

Doesn't 'git clone' download a copy of the whole repository?
> I thought that is what it did.
>

When you do a "git checkout" of an existing git repository it will
hard-link the files in .git/objects so that they are shared between the
repositories. Thus, the repository itself will not take any extra space,
only the actual work files themselves.

Regards,
Elias

[-- Attachment #2: Type: text/html, Size: 827 bytes --]

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> Richard Stallman <rms@gnu.org> writes:

 >> I don't use it myself, as 'git clone' is plenty fast and avoids some
 >> problems when multiple branches trying to update the same
 >> repository.

 > Doesn't 'git clone' download a copy of the whole repository?
 > I thought that is what it did.

	When applied to a /local/ repository, – it doesn’t.  Say, the
	following will clone an existing emacs-1 working directory into
	a new emacs-2 one, also setting up .git/objects/info/alternates
	under the latter pointing to emacs-1/.git/objects, so that all
	the Git commits available under the first are also available
	under the second.

$ git clone --shared emacs-1 emacs-2 

	The branches and tags of these two working copies are entirely
	independent, however.

	The --shared option is the default when both the source and the
	target directories are on the same filesystem.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> Ivan Shmakov <ivan@siamics.net> writes:

[…]

 > also setting up .git/objects/info/alternates under the latter
 > pointing to emacs-1/.git/objects, so that all the Git commits
 > available under the first are also available under the second.

 > $ git clone --shared emacs-1 emacs-2

 > The branches and tags of these two working copies are entirely
 > independent, however.

 > The --shared option is the default when both the source and the
 > target directories are on the same filesystem.

	I stand corrected, – the default in this case is to use
	hardlinks for the Git objects; no ‘alternates’ file is involved.

	Using --shared may be risky, as Git may choose to GC the
	dangling objects away from emacs-1, not taking into account the
	possibility of them being used in some other trees.  In my case,
	the source directory is often a “bare” Git repository used only
	to mirror the upstream one, and thus it has no “local” commits,
	which are somewhat likely to become dangling after rebases and
	such.  (And the upstream branches these mirrors track are
	fast-forward, so git-fetch(1) isn’t going to produce any
	dangling objects, either.)

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I wrote

  > Doesn't 'git clone' download a copy of the whole repository?
  > > I thought that is what it did.

and you answered

  > When you do a "git checkout" of an existing git repository it will
  > hard-link the files in .git/objects so that they are shared between the
  > repositories.

How does one get from the one t the other?  I am lost.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

Re: Multiple checkout copies

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

   > $ git clone --shared emacs-1 emacs-2

sounded like just the right thing until you posted
the caveat.  After this, I hesitate to use it:

	  > Using --shared may be risky, as Git may choose to GC the
	  > dangling objects away from emacs-1, not taking into account the
	  > possibility of them being used in some other trees.  In my case,
	  > the source directory is often a “bare” Git repository used only
	  > to mirror the upstream one, and thus it has no “local” commits,
	  > which are somewhat likely to become dangling after rebases and
	  > such.

Maybe I could use it in that way, but is there no way to make two
working trees both checked out in parallel from the same repository?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

Re: Multiple checkout copies

[Paul Eggert]

Richard Stallman wrote:
> How does one get from the one t the other?  I am lost.

For the first copy, you need to get a local copy of the repository, something I 
assume you've already done.  Something like this:

$ git clone rms@git.sv.gnu.org:/srv/git/emacs.git master
Cloning into 'master'...
remote: Counting objects: 721802, done.
remote: Compressing objects: 100% (133955/133955), done.
remote: Total 721802 (delta 589941), reused 718606 (delta 587176)
Receiving objects: 100% (721802/721802), 191.46 MiB | 4.04 MiB/s, done.
Resolving deltas: 100% (589941/589941), done.
Checking connectivity... done.

This first step is expensive, since it copies the repository across the network 
into the subdirectory 'master', and creates working files there too.  On my home 
GNU/Linux desktop, this took 4.6 minutes and the resulting directory consumed 
365 MB of disk space (222 MB for the repository and 144 MB for the working files).

For the second copy, you clone the repository locally, like this:

$ git clone master devel
Cloning into 'devel'...
done.

This second step is cheap: it took less than a second on my desktop.  It copies 
working files into the subdirectory 'devel', but it hard-links the repository 
rather than copying it, so it consumes only 144 MB (i.e., enough for the second 
copy of the working files).

Although the two subdirectories have logically independent repositories, the 
second repository costs almost nothing to create.

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> Richard Stallman <rms@gnu.org> writes:

[…]

 >> Using --shared may be risky, as Git may choose to GC the dangling
 >> objects away from emacs-1, not taking into account the possibility
 >> of them being used in some other trees.  In my case, the source
 >> directory is often a “bare” Git repository used only to mirror the
 >> upstream one, and thus it has no “local” commits, which are somewhat
 >> likely to become dangling after rebases and such.

 > Maybe I could use it in that way, but is there no way to make two
 > working trees both checked out in parallel from the same repository?

	I’m unsure I understand the purpose.

	The problem is that a Git repository (as in: working-copy/.git)
	is not just the graph of its respective commits, but also the
	mapping of branch (and tag) names to commit identifiers.  When a
	working copy is updated (say, via $ git pull), the respective
	branch name (.git/refs/heads/branchname) is also updated to
	point to the new head commit of the branch.  And I doubt that
	sharing such a mapping among several working copies for the same
	branch would be sensible.

	The approach I use is roughly as follows.

	First, I create a local clone of the remote repository /without/
	an associated working copy, like:

$ git clone --bare -- \
      git://example.org/jrh/example.git ~/public/download/git/example-2015.git 

	The repository so created can be updated with git-fetch(1):

$ GIT_DIR=~/public/download/git/example-2015.git \
      git fetch -t origin master:master 

	… Or without giving the branches explicitly, should
	example-2015.git/config contain an appropriate default, like:

[remote "origin"]
    url = git://example.org/jrh/example.git
    fetch = master:master example-42:example-42

	The working copies may now be created from this local repository
	(which git-clone(1) automatically aliases to ‘origin’)
	with --shared, and updated with the regular $ git pull command
	from there:

$ git clone --shared -- \
      ~/public/download/git/example-2015.git ~/devel/example-1 
$ git clone --shared -- \
      ~/public/download/git/example-2015.git ~/devel/example-2 
…
$ GIT_DIR=~/public/download/git/example-2015.git \
      git fetch -t origin master:master ## update the “bare” repository 
$ cd ~/devel/example-1 
$ git pull origin   ## update the working copy 

	This way, the commits which made it to the upstream are always
	available from example-2015.git and get reused upon git-pull(1),
	thus saving both time and filesystem space.  Furthermore, unless
	the upstream decides to “rewrite history” (at which point Git
	will require manual intervention), these commits won’t be
	eligible to GC, thus eliminating the risks generally associated
	with --shared.

	The commits made locally to either of the working copies stay in
	that same working copy (unless explicitly copied.)  I’m unsure
	that those of them which get incorporated upstream (and thus
	become available from example-2015.git) would be eligible for
	GC, but any “temporary” histories held within the individual
	example-N/.git (such as those left behind after a successful
	$ git rebase) surely would.

	The .git/refs/ mappings of these working copies are also
	entirely independent.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> I wrote
>
>   > Doesn't 'git clone' download a copy of the whole repository?
>   > > I thought that is what it did.
>
> and you answered
>
>   > When you do a "git checkout" of an existing git repository it will
>   > hard-link the files in .git/objects so that they are shared between the
>   > repositories.
>
> How does one get from the one t the other?  I am lost.

I suppose by fixing a finger fluke.  The second statement can be turned
from nonsensical to accurate by replacing "git checkout" with "git
clone".  Of course, you still need to make this "of an existing git
repository on the same file system" since otherwise you won't be getting
hard links.

-- 
David Kastrup

Re: Multiple checkout copies

[Achim Gratz]

Richard Stallman writes:
>    > $ git clone --shared emacs-1 emacs-2
>
> sounded like just the right thing until you posted
> the caveat.  After this, I hesitate to use it:
>
> 	  > Using --shared may be risky, as Git may choose to GC the
> 	  > dangling objects away from emacs-1, not taking into account the
> 	  > possibility of them being used in some other trees.  In my case,
> 	  > the source directory is often a “bare” Git repository used only
> 	  > to mirror the upstream one, and thus it has no “local” commits,
> 	  > which are somewhat likely to become dangling after rebases and
> 	  > such.

That's mostly a red herring.  The objects shared are hard-linked, so
even if the GC removes objects in a repository that doesn't reference
them anymore they continue to exist in the repository that still needs
them.  As long as you do not delete that other repository you cannot
lose anything.

> Maybe I could use it in that way, but is there no way to make two
> working trees both checked out in parallel from the same repository?

The officially sanctioned Git way of getting a new worktree is to make
another clone.  If you don't intend to use it as a full repository later
on, you can do a shallow clone or you can use another repository as an
"alternate" to skip copying / linking the object files.

There is a sort-of way of splitting off additional working trees, either
using the --work-tree argument or by setting the GIT_WORK_TREE
environment variable or even manipulating the core.worktree property of
the repository.  But you can only ever make changes on a single worktree
anyway since a lot of meta-information regarding the worktree is also
kept in the repository, so while you have those multiple working
directories, only the last of them is properly "checked out".

The way to do this properly is to link all shared information back to
the original repository (I'd only ever use a bare one for this) and keep
the per-worktree files separated.  The git-new-workdir script does this,
but it's still in contrib since it doesn't work on some systems.

http://git.kernel.org/cgit/git/git.git/tree/contrib/workdir/git-new-workdir

You are quite likely on a system where that script works, so it would
just be a matter of dropping it into /usr/lib/git and making it
executable.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Multiple checkout copies

[David Kastrup]

Achim Gratz <Stromeko@nexgo.de> writes:

> Richard Stallman writes:
>>    > $ git clone --shared emacs-1 emacs-2
>>
>> sounded like just the right thing until you posted
>> the caveat.  After this, I hesitate to use it:
>>
>> 	  > Using --shared may be risky, as Git may choose to GC the
>> 	  > dangling objects away from emacs-1, not taking into account the
>> 	  > possibility of them being used in some other trees.  In my case,
>> 	  > the source directory is often a “bare” Git repository used only
>> 	  > to mirror the upstream one, and thus it has no “local” commits,
>> 	  > which are somewhat likely to become dangling after rebases and
>> 	  > such.
>
> That's mostly a red herring.  The objects shared are hard-linked,

No, they aren't.

       --shared, -s
           When the repository to clone is on the local machine, instead of
           using hard links, automatically setup .git/objects/info/alternates
           to share the objects with the source repository. The resulting
           repository starts out without any object of its own.

And that's what makes this operation potentially dangerous.  --shared
has the advantage of never needing additional object space (the hard
links used otherwise will start out with just requiring additional
inodes, but as soon as objects get repacked, their contents are present
twice as well), and being effective even when the repository is not on
the same file system.

The manual explicitly states with regard to --shared

           NOTE: this is a possibly dangerous operation; do not use it
           unless you understand what it does.

And a corollary to "not not use it unless you understand what it does"
is "do not publish any recipes or workflows containing --shared" because
recipes or workflows are intended for the benefit of people not
understanding every step.

> so even if the GC removes objects in a repository that doesn't
> reference them anymore they continue to exist in the repository that
> still needs them.  As long as you do not delete that other repository
> you cannot lose anything.

Apparently you confuse --shared with _not_ using --shared.

-- 
David Kastrup

Re: Multiple checkout copies

[Achim Gratz]

David Kastrup writes:
> Apparently you confuse --shared with _not_ using --shared.

Indeed, I was reading "--local" instead of "--shared", sorry. :-P

In any case, as long as the repo cloned via "--shared" is a bare
repository that was cloned from upstream Emacs and you do not clone the
second generation again in this manner, "--shared" is still safe.  The
possible copy of objects when repack operations are run makes this less
useful as a space-saving measure than one would think, however.  It can
still be a net win for throw-away clones needed during a build.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Multiple checkout copies

[David Kastrup]

Achim Gratz <Stromeko@nexgo.de> writes:

> David Kastrup writes:
>> Apparently you confuse --shared with _not_ using --shared.
>
> Indeed, I was reading "--local" instead of "--shared", sorry. :-P
>
> In any case, as long as the repo cloned via "--shared" is a bare
> repository that was cloned from upstream Emacs and you do not clone the
> second generation again in this manner, "--shared" is still safe.

No, it isn't.  You can have both local branches and remote branches
removed from the cloned bare repo while your --shared clone still uses
objects that are no longer retained in the first clone.

> It can still be a net win for throw-away clones needed during a build.

Basically the only sensible use is for temporary clones used (and
deleted again) within a script.

-- 
David Kastrup

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> David Kastrup <dak@gnu.org> writes:
>>>>> Achim Gratz <Stromeko@nexgo.de> writes:

[…]

 >> In any case, as long as the repo cloned via "--shared" is a bare
 >> repository that was cloned from upstream Emacs and you do not clone
 >> the second generation again in this manner, "--shared" is still
 >> safe.

 > No, it isn't.  You can have both local branches and remote branches
 > removed from the cloned bare repo while your --shared clone still
 > uses objects that are no longer retained in the first clone.

	“Can” in the sense that you /can/ use Git to shoot yourself in
	the foot, or are there cases where git-fetch(1) would indeed
	autoremove any branch references from a bare repository?

 >> It can still be a net win for throw-away clones needed during a
 >> build.

 > Basically the only sensible use is for temporary clones used (and
 > deleted again) within a script.

	I use a separate filesystem for local mirrors, including those
	of upstream Git repositories.  Thus, --shared is pretty much the
	only space-saving option.

	Works for me that way since c. 2008.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[David Kastrup]

Ivan Shmakov <ivan@siamics.net> writes:

>>>>>> David Kastrup <dak@gnu.org> writes:
>>>>>> Achim Gratz <Stromeko@nexgo.de> writes:
>
> […]
>
>  >> In any case, as long as the repo cloned via "--shared" is a bare
>  >> repository that was cloned from upstream Emacs and you do not clone
>  >> the second generation again in this manner, "--shared" is still
>  >> safe.
>
>  > No, it isn't.  You can have both local branches and remote branches
>  > removed from the cloned bare repo while your --shared clone still
>  > uses objects that are no longer retained in the first clone.
>
> 	“Can” in the sense that you /can/ use Git to shoot yourself in
> 	the foot, or are there cases where git-fetch(1) would indeed
> 	autoremove any branch references from a bare repository?

If you want to mirror the upstream, you'll use "git fetch -p" in order
to have branches that are removed upstream to get deleted in the mirror
as well.  Even without -p, references in frequently rewritten work
branches (like Git's pu or next branches) will disappear eventually.

>  >> It can still be a net win for throw-away clones needed during a
>  >> build.
>
>  > Basically the only sensible use is for temporary clones used (and
>  > deleted again) within a script.
>
> 	I use a separate filesystem for local mirrors, including those
> 	of upstream Git repositories.  Thus, --shared is pretty much the
> 	only space-saving option.
>
> 	Works for me that way since c. 2008.

"Works for me" is subject to "until it doesn't".  Which is the whole
point about warning about potentially dangerous operations.

-- 
David Kastrup

Re: Multiple checkout copies

[Steinar Bang]

>>>>> Richard Stallman <rms@gnu.org>:

> Using git, is there a way to check out copies of different branches in
> parallel (in different directories) from the same clone of the
> repository?

If you're not concerned with disk usage, the simplest way is something
like this (assuming you're placed in the directory above where the
current emacs is checked out as "emacs"):
 cp -a emacs emacs-24
 cd emacs-24
 git checkout emacs-24

A slightly different version of the above will save disk space by using
hard links to files in the original repository:
 git clone emacs emacs-24
 cd emacs-24
 git checkout emacs-24

(most of the repository is immutable, and it is the immutable stuff that
is hardlinked, so these two repositories will diverge over time, so the
space save is just initial)

Note: neither of these approaches will let you share stuff between the
two clones without going by the remote server (however there are ways to
do so, if this is of interest).

Re: Multiple checkout copies

[David Kastrup]

Steinar Bang <sb@dod.no> writes:

>>>>>> Richard Stallman <rms@gnu.org>:
>
>> Using git, is there a way to check out copies of different branches in
>> parallel (in different directories) from the same clone of the
>> repository?
>
> If you're not concerned with disk usage, the simplest way is something
> like this (assuming you're placed in the directory above where the
> current emacs is checked out as "emacs"):
>  cp -a emacs emacs-24
>  cd emacs-24
>  git checkout emacs-24
>
> A slightly different version of the above will save disk space by using
> hard links to files in the original repository:
>  git clone emacs emacs-24
>  cd emacs-24
>  git checkout emacs-24
>
> (most of the repository is immutable, and it is the immutable stuff that
> is hardlinked, so these two repositories will diverge over time, so the
> space save is just initial)
>
> Note: neither of these approaches will let you share stuff between the
> two clones without going by the remote server (however there are ways to
> do so, if this is of interest).

Sigh.  The second method (creating a clone of the repo) will make this
clone have the original repository as its upstream, not the remote one.
So the second approach will _only_ let you share stuff by going through
the first repository.

-- 
David Kastrup

Re: Multiple checkout copies

[Stefan Monnier]

> Maybe I could use it in that way, but is there no way to make two
> working trees both checked out in parallel from the same repository?

If you mean "same remote repository", then you can simply "cp -a" and
that will work fine (tho there are many other ways to do it).

If you mean "same local repository", then the only option is
git-new-workdir (which is what I use).


        Stefan

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> David Kastrup <dak@gnu.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:
>>>>> David Kastrup <dak@gnu.org> writes:

[…]

 >>> No, it isn't.  You can have both local branches and remote branches
 >>> removed from the cloned bare repo while your --shared clone still
 >>> uses objects that are no longer retained in the first clone.

 >> “Can” in the sense that you /can/ use Git to shoot yourself in the
 >> foot, or are there cases where git-fetch(1) would indeed autoremove
 >> any branch references from a bare repository?

 > If you want to mirror the upstream, you'll use "git fetch -p" in
 > order to have branches that are removed upstream to get deleted in
 > the mirror as well.

	And why would I want that, especially given that I’m already
	aware that a. some of the objects may be used elsewhere (and
	thus such an operation would be potentially unsafe) and b. the
	packs received from the remote are likely to contain a mix of
	Git objects belonging to both removed and extant branches, and
	it would thus /not/ be possible to remove them anyway?

	And as for repacking such a mirror from time to time, — it’s
	going to be a sure nuisance due to backup reasons, etc.

 > Even without -p, references in frequently rewritten work branches
 > (like Git's pu or next branches) will disappear eventually.

	Do fast-forward updates (which git-fetch(1) defaults to) ever
	result in dangling Git objects?

	Also, I’m typically interested in mirroring just a few branches
	(mostly ‘master’ and the latest stable, if any.)  Per my
	experience, such branches rarely (if ever) get “rewritten.”

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[David Kastrup]

Ivan Shmakov <ivan@siamics.net> writes:

>>>>>> David Kastrup <dak@gnu.org> writes:
>>>>>> Ivan Shmakov <ivan@siamics.net> writes:
>>>>>> David Kastrup <dak@gnu.org> writes:
>
> […]
>
>  >>> No, it isn't.  You can have both local branches and remote branches
>  >>> removed from the cloned bare repo while your --shared clone still
>  >>> uses objects that are no longer retained in the first clone.
>
>  >> “Can” in the sense that you /can/ use Git to shoot yourself in the
>  >> foot, or are there cases where git-fetch(1) would indeed autoremove
>  >> any branch references from a bare repository?
>
>  > If you want to mirror the upstream, you'll use "git fetch -p" in
>  > order to have branches that are removed upstream to get deleted in
>  > the mirror as well.
>
> 	And why would I want that,

In order not to accumulate temporary branches that are no longer present
in upstream?

> 	especially given that I’m already aware that a. some of the
> 	objects may be used elsewhere (and thus such an operation would
> 	be potentially unsafe) and b. the packs received from the remote
> 	are likely to contain a mix of Git objects belonging to both
> 	removed and extant branches,

Why would they?

> 	and it would thus /not/ be possible to remove them anyway?
>
> 	And as for repacking such a mirror from time to time, — it’s
> 	going to be a sure nuisance due to backup reasons, etc.

Uh, it's going to get repacked anyway.  Git does that without asking.

>  > Even without -p, references in frequently rewritten work branches
>  > (like Git's pu or next branches) will disappear eventually.
>
> 	Do fast-forward updates (which git-fetch(1) defaults to) ever
> 	result in dangling Git objects?

Maybe it would be worth checking the man pages before making such
statements.  It's just annoying when people spray arbitrary statements
to the list and leave it to others to clean up.

git-fetch does not merge in any manner and consequently also does not
"fast-forward".  It updates references after making them backed by the
respective objects.  You are confusing this with the git-pull action of
updating a local branch based on a remote-tracking branch.

There is a very limited situation when using explicit branch names for
source and target branch where git-fetch will only update a reference to
a non-descendant when given the option -f.  Since there is no way to
trigger a "slow forward" (also known as "merge") without involving the
work directory and index, this is neither the same as a fast forward,
nor referenced as such.

> 	Also, I’m typically interested in mirroring just a few branches
> 	(mostly ‘master’ and the latest stable, if any.)  Per my
> 	experience, such branches rarely (if ever) get “rewritten.”

"I'm typically interested in" is no base for promoting problematic
workflows since it violates the "do not use it unless you understand
what it does" dictum for the recipient of such a recipe.

-- 
David Kastrup

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> David Kastrup <dak@gnu.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:
>>>>> David Kastrup <dak@gnu.org> writes:

[…]

 >>> If you want to mirror the upstream, you'll use "git fetch -p" in
 >>> order to have branches that are removed upstream to get deleted in
 >>> the mirror as well.

 >> And why would I want that,

 > In order not to accumulate temporary branches that are no longer
 > present in upstream?

	How would the branches “accumulate” in the case git-fetch(1) is
	given an explicit list of them to follow?

 >> especially given that I’m already aware that a. some of the objects
 >> may be used elsewhere (and thus such an operation would be
 >> potentially unsafe) and b. the packs received from the remote are
 >> likely to contain a mix of Git objects belonging to both removed and
 >> extant branches,

 > Why would they?

	Because the development happens in several branches in parallel?

 >> and it would thus /not/ be possible to remove them anyway?

 >> And as for repacking such a mirror from time to time, — it’s going
 >> to be a sure nuisance due to backup reasons, etc.

 > Uh, it's going to get repacked anyway.  Git does that without asking.

	… Unless disabled with gc.auto = 0, gc.autopacklimit = 0.

 >>> Even without -p, references in frequently rewritten work branches
 >>> (like Git's pu or next branches) will disappear eventually.

 >> Do fast-forward updates (which git-fetch(1) defaults to) ever result
 >> in dangling Git objects?

 > Maybe it would be worth checking the man pages before making such
 > statements.  It's just annoying when people spray arbitrary
 > statements to the list and leave it to others to clean up.

 > git-fetch does not merge in any manner and consequently also does not
 > "fast-forward".  It updates references after making them backed by
 > the respective objects.  You are confusing this with the git-pull
 > action of updating a local branch based on a remote-tracking branch.

	Well, would you then care to explain the wording of the Git
	error message below?

+ git fetch -t origin master:master
From git://github.com/XXX/XXX
 ! [rejected]        master     -> master  (non-fast-forward)

	Not to mention that git-fetch(1) explicitly uses this same term
	(as of Git 2.1.4.)

	I do not generally support the opinion that Git terminology is
	confusing, but I find it quite possible that it may sometimes be
	inconsistent at best.  Maybe it’s just such a case.

 > There is a very limited situation when using explicit branch names
 > for source and target branch where git-fetch will only update a
 > reference to a non-descendant when given the option -f.

	Yes.

[…]

 >> Also, I’m typically interested in mirroring just a few branches
 >> (mostly ‘master’ and the latest stable, if any.)  Per my experience,
 >> such branches rarely (if ever) get “rewritten.”

 > "I'm typically interested in" is no base for promoting problematic
 > workflows since it violates the "do not use it unless you understand
 > what it does" dictum for the recipient of such a recipe.

	The whole idea behind this discussion is to clarify when
	--shared is reasonable, — and when it isn’t.  Isn’t it?

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[David Kastrup]

Ivan Shmakov <ivan@siamics.net> writes:

>>>>>> David Kastrup <dak@gnu.org> writes:
>
>  > "I'm typically interested in" is no base for promoting problematic
>  > workflows since it violates the "do not use it unless you understand
>  > what it does" dictum for the recipient of such a recipe.
>
> 	The whole idea behind this discussion is to clarify when
> 	--shared is reasonable, — and when it isn’t.  Isn’t it?

And the manual page clearly states that it requires to know what you are
doing.  That makes it feasible for use in a script, and unfeasible for
use in a recipe: the latter _can_ and will be varied, violating the "do
not use it unless you understand what it does" prerequisite given in its
manual page.

And all of the handwaving and qualifications _after_ having the problems
pointed out make _very_ obvious that the required information for using
it safely is not likely to be communicated in informal discussions.

In addition, we've had --shared confused with --local (which is implied
anyway).  So the takeaway is not to use any options in connection with
creating local clones manually.  That steers around all of the not
reliably reproducible problems, at a mostly moderate cost in disk usage.

"Handling a foot gun is not hard and can be fun" is a frequent software
engineering pain vector.

-- 
David Kastrup

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> David Kastrup <dak@gnu.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:
>>>>> David Kastrup <dak@gnu.org> writes:

 >>> "I'm typically interested in" is no base for promoting problematic
 >>> workflows since it violates the "do not use it unless you
 >>> understand what it does" dictum for the recipient of such a recipe.

 >> The whole idea behind this discussion is to clarify when --shared is
 >> reasonable, — and when it isn’t.  Isn’t it?

 > And the manual page clearly states that it requires to know what you
 > are doing.  That makes it feasible for use in a script, and
 > unfeasible for use in a recipe: the latter _can_ and will be varied,
 > violating the "do not use it unless you understand what it does"
 > prerequisite given in its manual page.

 > And all of the handwaving and qualifications _after_ having the
 > problems pointed out make _very_ obvious that the required
 > information for using it safely is not likely to be communicated in
 > informal discussions.

	I’m getting an impression that the above should make me feeling
	guilt or ashamed of sharing my workflow with the audience.

	But I still do not, nor see the reason to.

 > In addition, we've had --shared confused with --local (which is
 > implied anyway).

	FTR, in my initial message, I’ve indeed confused --shared with
	--local as the /default/ for git-clone(1), and even corrected
	myself [1].  I didn’t confuse the /effects/ of these options.

 > So the takeaway is not to use any options in connection with creating
 > local clones manually.

	I disagree.

 > That steers around all of the not reliably reproducible problems, at
 > a mostly moderate cost in disk usage.

 > "Handling a foot gun is not hard and can be fun" is a frequent
 > software engineering pain vector.

	In this discussion, I saw no scenario under which my approach,
	as given in [1], would fail (but surely would appreciate any.)

	It wasn’t my intent to make claims regarding the use of --shared
	outside of that particular approach.

[1] news:87oapcgp2c.fsf@violet.siamics.net
    http://permalink.gmane.org/gmane.emacs.devel/182278

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[Steinar Bang]

>>>>> David Kastrup <dak@gnu.org>:

> Sigh.  The second method (creating a clone of the repo) will make this
> clone have the original repository as its upstream, not the remote
> one.  So the second approach will _only_ let you share stuff by going
> through the first repository.

Right, I've never used it myself.  I just do "cp -a".

How about this, then (also untested):
  git clone emacs emacs-24
  cd emacs-24
  git remote set-url origin git://git.savannah.gnu.org/emacs.git
  git checkout emacs-24

(personally I'll stay with the "cp -a"...)

Re: Multiple checkout copies

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > A slightly different version of the above will save disk space by using
  > hard links to files in the original repository:
  >  git clone emacs emacs-24
  >  cd emacs-24
  >  git checkout emacs-24

Thanks.  Maybe that is what I want to do.

  > Sigh.  The second method (creating a clone of the repo) will make this
  > clone have the original repository as its upstream, not the remote one.
  > So the second approach will _only_ let you share stuff by going through
  > the first repository.

My idea is to do my editing in the 'emacs' directory, as well as
pulling updates there.  So these points seem like features rather than
bugs.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

Re: Multiple checkout copies

[Achim Gratz]

Richard Stallman writes:
>   > A slightly different version of the above will save disk space by using
>   > hard links to files in the original repository:
>   >  git clone emacs emacs-24
>   >  cd emacs-24
>   >  git checkout emacs-24
>
> Thanks.  Maybe that is what I want to do.
>
> My idea is to do my editing in the 'emacs' directory, as well as
> pulling updates there.  So these points seem like features rather than
> bugs.

The safest (but a bit more long-winded) way of dealing with simultaneous
work on multiple branches is IMHO this setup (per example doing work on
master and emacs-24):

$ git clone git://git.savannah.gnu.org/emacs.git --bare emacs.git
$ git clone emacs.git/ emacs-master
$ git clone emacs.git/ emacs-24
$ cd emacs-24 ; git checkout emacs-24 ; cd ..

This way there's only a single repository to push upstream from (which
also makes it a bit easier to check what you intend to push).  Making
this bare ensures you don't do any other work on this repo.  You only
need to be connected to the internet for updating this repo and can
air-gap otherwise (or work through a firewall).  Last but not least, all
clones made from that single bare repo all have the same configuration,
so you don't need to remember which one is upstream of the other.

The downside is that you must always update in two steps when pulling
(emacs.git from savannah, then the working repo from emacs.git).
However, this is easily scripted or made an alias.  Pushing also has to
be done in two steps, just the other way around.  I personally see this
as an advantage, since it gives you another check-point before things go
public and become essentially immutable.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: Multiple checkout copies

[David Kastrup]

Achim Gratz <Stromeko@nexgo.de> writes:

> Richard Stallman writes:
>>   > A slightly different version of the above will save disk space by using
>>   > hard links to files in the original repository:
>>   >  git clone emacs emacs-24
>>   >  cd emacs-24
>>   >  git checkout emacs-24
>>
>> Thanks.  Maybe that is what I want to do.
>>
>> My idea is to do my editing in the 'emacs' directory, as well as
>> pulling updates there.  So these points seem like features rather than
>> bugs.
>
> The safest (but a bit more long-winded) way of dealing with simultaneous
> work on multiple branches is IMHO this setup (per example doing work on
> master and emacs-24):
>
> $ git clone git://git.savannah.gnu.org/emacs.git --bare emacs.git
> $ git clone emacs.git/ emacs-master
> $ git clone emacs.git/ emacs-24
> $ cd emacs-24 ; git checkout emacs-24 ; cd ..
>
> This way there's only a single repository to push upstream from (which
> also makes it a bit easier to check what you intend to push).  Making
> this bare ensures you don't do any other work on this repo.

Minus the bare repository (I use my main repository) this is basically
what I do when I find I need to work on a different branch temporarily.
The difference is that I create my cloned directories on-demand and
remove them again once the work has been pushed upstream or integrated
wherever it is supposed to go.

While it isn't as formal a way to work as having one dedicated bare
repository, it is easy to do ad-hoc without prior setup.  Yes,
ping-ponging changes through the main repository is a bit of a nuisance.

-- 
David Kastrup

Re: Multiple checkout copies

[Ivan Shmakov]

>>>>> Achim Gratz <Stromeko@nexgo.de> writes:
>>>>> Richard Stallman writes:

[…]

 >> My idea is to do my editing in the 'emacs' directory, as well as
 >> pulling updates there.  So these points seem like features rather
 >> than bugs.

[…]

 > $ git clone git://git.savannah.gnu.org/emacs.git --bare emacs.git
 > $ git clone emacs.git/ emacs-master
 > $ git clone emacs.git/ emacs-24
 > $ cd emacs-24 ; git checkout emacs-24 ; cd ..

[…]

 > The downside is that you must always update in two steps when pulling
 > (emacs.git from savannah, then the working repo from emacs.git).
 > However, this is easily scripted or made an alias.

	Yes.

 > Pushing also has to be done in two steps, just the other way around.

	Actually, it isn’t strictly necessary, as it’s possible to give
	the target repository URI explicitly to git-push(1), like:

$ cd emacs-master 
$ git push -n git+ssh://USERNAME@git.sv.gnu.org/srv/git/emacs.git \
      HEAD:master ; ## double check here 
$ git push    git+ssh://USERNAME@git.sv.gnu.org/srv/git/emacs.git \
      HEAD:master 

 > I personally see this as an advantage, since it gives you another
 > check-point before things go public and become essentially immutable.

	I prefer to keep my bare repositories as just mirrors of the
	upstream ones, not cluttered with any local commits I may have.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Multiple checkout copies

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I am pretty sure I don't want to use --shared.
Could we take the dispute about it off this list?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.