From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Bob Proulx Newsgroups: gmane.emacs.devel Subject: Re: Move to git is imminent - awaiting Stefan's approval Date: Sat, 11 Jan 2014 23:27:22 -0700 Message-ID: <20140112062722.GA15615@hysteria.proulx.com> References: <8761pvvoq1.fsf@rosalinde.fritz.box> <83d2k3sqqz.fsf@gnu.org> <52CC2D45.1000908@alice.it> <83a9f7sosr.fsf@gnu.org> <52CC5ED3.2000903@alice.it> <83d2k2fkg1.fsf@gnu.org> <87txdenwz3.fsf@igel.home> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1389508054 13668 80.91.229.3 (12 Jan 2014 06:27:34 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 12 Jan 2014 06:27:34 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Jan 12 07:27:39 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1W2EWI-0003wZ-MU for ged-emacs-devel@m.gmane.org; Sun, 12 Jan 2014 07:27:38 +0100 Original-Received: from localhost ([::1]:36642 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W2EWI-0000TY-8p for ged-emacs-devel@m.gmane.org; Sun, 12 Jan 2014 01:27:38 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57288) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W2EW9-0000Sv-5x for emacs-devel@gnu.org; Sun, 12 Jan 2014 01:27:35 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W2EW3-0001J6-6T for emacs-devel@gnu.org; Sun, 12 Jan 2014 01:27:29 -0500 Original-Received: from joseki.proulx.com ([216.17.153.58]:54690) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W2EW2-0001J2-VH for emacs-devel@gnu.org; Sun, 12 Jan 2014 01:27:23 -0500 Original-Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 5425521227 for ; Sat, 11 Jan 2014 23:27:22 -0700 (MST) Original-Received: by hysteria.proulx.com (Postfix, from userid 1000) id 24F4B2DCD0; Sat, 11 Jan 2014 23:27:22 -0700 (MST) Content-Disposition: inline In-Reply-To: <87txdenwz3.fsf@igel.home> User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 216.17.153.58 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:168159 Archived-At: Andreas Schwab wrote: > Eli Zaretskii writes: > > So I guess we will need to repack the repository on savannah as a > > prerequisite for switching, and then make sure it is repacked > > regularly afterwards. > > I think what should be done is to increase the default configuration > value of pack.window for the git repos on savannah. That should help > keeping the packs better packed in the long run. All of the repositories are repacked monthly on the 4th. As you can see it is a heavy operation and therefore not run every day. But this thread seems to have exposed a deficiency in the current repacking. Currently the script does: git --git-dir=$dir gc -q Obvious from the discussion here that is not sufficient. However: Angelo Graziosi wrote: > $ git gc --aggressive > Counting objects: 735546, done. > Delta compression using up to 2 threads. > Compressing objects: 100% (734446/734446), done. > Writing objects: 100% (735546/735546), done. > Total 735546 (delta 589814), reused 143177 (delta 0) > Checking connectivity: 735546, done. > > which took almost 3 hours to be completed. Eli Zaretskii wrote: > And now I understand why git ran out of memory on Windows XP: the > memory footprint of git-pack-objects goes up to 3GB, whereas Windows > XP has only 3GB address space for all the applications. Three hours and 3G of memory for this one repack! I worry that turning --agressive on globally will cause the server to fall over. Also apparently careful tuning of threads is needed. However the disk and network savings look significant. I have a couple of questions if someone already has the clone available for a test. Does a clone from the agressively repacked to 327M repository remain similarly compacted when cloned again from it? If an agressively repacked repository is again repacked but this time without the --agressive option does the size stay around 327M or does it get expanded on the subsequent pass? (Wondering if we can periodically run 'git gc --agressive' on the larger git repositories at a niced background task priority but not too often and still achieve a good benefit for the time between agressive repacks.) Bob