From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Bob Proulx Newsgroups: gmane.emacs.devel Subject: Re: Move to git is imminent - awaiting Stefan's approval Date: Sun, 12 Jan 2014 14:13:28 -0700 Message-ID: <20140112211328.GA2537@hysteria.proulx.com> References: <83d2k3sqqz.fsf@gnu.org> <52CC2D45.1000908@alice.it> <83a9f7sosr.fsf@gnu.org> <52CC5ED3.2000903@alice.it> <83d2k2fkg1.fsf@gnu.org> <87txdenwz3.fsf@igel.home> <20140112062722.GA15615@hysteria.proulx.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1389561216 15372 80.91.229.3 (12 Jan 2014 21:13:36 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 12 Jan 2014 21:13:36 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Jan 12 22:13:44 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1W2SLn-0004l4-2T for ged-emacs-devel@m.gmane.org; Sun, 12 Jan 2014 22:13:43 +0100 Original-Received: from localhost ([::1]:39497 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W2SLm-0008HV-JE for ged-emacs-devel@m.gmane.org; Sun, 12 Jan 2014 16:13:42 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60812) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W2SLg-0008FV-8W for emacs-devel@gnu.org; Sun, 12 Jan 2014 16:13:40 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W2SLb-0000VN-S8 for emacs-devel@gnu.org; Sun, 12 Jan 2014 16:13:36 -0500 Original-Received: from joseki.proulx.com ([216.17.153.58]:57706) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W2SLa-0000VB-Rc for emacs-devel@gnu.org; Sun, 12 Jan 2014 16:13:31 -0500 Original-Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 919DD21227 for ; Sun, 12 Jan 2014 14:13:28 -0700 (MST) Original-Received: by hysteria.proulx.com (Postfix, from userid 1000) id 808562DCD2; Sun, 12 Jan 2014 14:13:28 -0700 (MST) Mail-Followup-To: emacs-devel@gnu.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 216.17.153.58 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:168247 Archived-At: Andreas Schwab wrote: > Bob Proulx writes: > > Three hours and 3G of memory for this one repack! I worry that > > turning --agressive on globally will cause the server to fall over. > > IMHO --aggressive is too aggressive, and not useful for on-going > maintenance. Incresing pack.window to 50 or so should already help > enough to improve the packing. It sounds like I should schedule a full agressive repack once. Once being the operative word. git repack -a -d -f --window=250 --depth=250 And then as you suggest above ensure that the regularly routine ongoing maintenance 'git gc' runs with a large enough pack window. I see that the git-gc default is --window=250 so at that point nothing more should need to be done. The normal 'git gc' routine maintenance should be okay at that point. Is that right? Seems like it. We can tweak this further as needed. Certain times of day the vcs system is quite heavily loaded. The vcs system is a VM on a dom0 also hosting many other VMs. At some times of day the entire dom0 is very heavily I/O limited. This has been an ongoing problem and discussion. It is definitely an ongoing problem. I will run this during a less busy dom0 time. I spent some time researching this problem and found this note and some information in the entire thread useful and interesting. It is a long thread but there are some good gems in there. http://article.gmane.org/gmane.comp.gcc.devel/94613 > > If an agressively repacked repository is again repacked but this time > > without the --agressive option does the size stay around 327M or does > > it get expanded on the subsequent pass? > > Unless you run repack with -f (ie. gc --aggresive) existing deltas are > reused, and only newly added objects are deltified. Sounds good. I will note that -f forces a full (--no-reuse-delta) repack of everything and should be avoided. > > (Wondering if we can periodically run 'git gc --agressive' on the > > larger git repositories at a niced background task priority but not > > too often and still achieve a good benefit for the time between > > agressive repacks.) > > Another option is to touch a .keep file for the largest pack so > that it is never touched again. New objects will then be added to a > separate pack even after git gc. If that large pack is already well > packed this should save some processing time. That seems like a useful additional tweak for a large stage such as this. If nothing else it will help out the backup since that file won't be changing on a routine basis and will remain static for the purposes of backup transfer. This could be applied to several of the large repositories. It seems that on the client side after a new clone that this tweak is not propagated. It seems that if there are multiple packs on the server side that they are combined into a single pack file. But without being repacked. Therefore downstream clients that wish this would need to do it manually after a clone. Bob