From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Compress VCS checkouts? Date: Mon, 05 Feb 2018 11:27:10 +0100 Message-ID: <87vafbhgu9.fsf@gnu.org> References: <20180202145011.6961c2c9@centurylink.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:32908) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eidzZ-0006G7-Di for guix-devel@gnu.org; Mon, 05 Feb 2018 05:27:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eidzV-0006Lp-Ev for guix-devel@gnu.org; Mon, 05 Feb 2018 05:27:17 -0500 Received: from hera.aquilenet.fr ([2a0c:e300::1]:51754) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eidzV-0006JG-8O for guix-devel@gnu.org; Mon, 05 Feb 2018 05:27:13 -0500 In-Reply-To: <20180202145011.6961c2c9@centurylink.net> (Eric Bavier's message of "Fri, 2 Feb 2018 14:50:11 -0600") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Eric Bavier Cc: guix-devel@gnu.org Hello! Eric Bavier skribis: > A while back I toyed with the idea of a "Sources DVD" for Guix, much > like Trisquel has. While looking into the feasability of this, I > noticed that the git/svn/cvs/hg checkouts our packages use are not > compressed in the store. So I wrote a simple script to compare the raw > size of all source checkouts with their corresponding xz-compressed > tarballs.=20=20 > > At the time, I couldn't convince myself the trouble was worth the > effort; we had only 385MiB of uncompressed source checkouts, which > would compress to 135MiB. > > Since that initial investigation, the number of packages using VCS > checkouts has risen from 50 to 389, and the on-disk size of that source > has risen to 3784MiB uncompressed. We could compress that down to > 538MiB. > > Compression in not free, but I think it would be nice for our build > servers who have to store and serve (which, iirc, requires them to > compress it anyhow) this source. =E2=80=98guix publish -C=E2=80=99 always compresses store items, unless the= y are regular files ending in =E2=80=9C.gz=E2=80=9D, =E2=80=9C.xz=E2=80=9D, etc. So ther= e=E2=80=99s no problem here. It seems to me that the only benefit of compressing checkouts would be the use case you=E2=80=99re talking about. For everyday use, it=E2=80=99s = actually more efficient to leave the uncompressed, so that people don=E2=80=99t end up do= ing round of compression/decompression when they run =E2=80=9Cguix build=E2=80= =9D. WDYT? That said, at worst, this could be addressed by writing a specific tool for the job. Ludo=E2=80=99.