From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Bavier Subject: Re: Compress VCS checkouts? Date: Mon, 5 Feb 2018 18:49:22 -0600 Message-ID: <20180205184922.3ff8e330@centurylink.net> References: <20180202145011.6961c2c9@centurylink.net> <87vafbhgu9.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:50852) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eirRy-0005RB-Db for guix-devel@gnu.org; Mon, 05 Feb 2018 19:49:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eirRu-0002pv-O5 for guix-devel@gnu.org; Mon, 05 Feb 2018 19:49:30 -0500 Received: from mail.onyx.syn-alias.com ([206.152.134.66]:41705 helo=smtp.centurylink.net) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eirRu-0002od-J9 for guix-devel@gnu.org; Mon, 05 Feb 2018 19:49:26 -0500 In-Reply-To: <87vafbhgu9.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ludovic =?UTF-8?B?Q291cnTDqHM=?= Cc: guix-devel@gnu.org On Mon, 05 Feb 2018 11:27:10 +0100 ludo@gnu.org (Ludovic Court=C3=A8s) wrote: > Hello! >=20 > Eric Bavier skribis: >=20 > > A while back I toyed with the idea of a "Sources DVD" for Guix, much > > like Trisquel has. While looking into the feasability of this, I > > noticed that the git/svn/cvs/hg checkouts our packages use are not > > compressed in the store. So I wrote a simple script to compare the raw > > size of all source checkouts with their corresponding xz-compressed > > tarballs. =20 > > > > At the time, I couldn't convince myself the trouble was worth the > > effort; we had only 385MiB of uncompressed source checkouts, which > > would compress to 135MiB. > > > > Since that initial investigation, the number of packages using VCS > > checkouts has risen from 50 to 389, and the on-disk size of that source > > has risen to 3784MiB uncompressed. We could compress that down to > > 538MiB. > > > > Compression in not free, but I think it would be nice for our build > > servers who have to store and serve (which, iirc, requires them to > > compress it anyhow) this source. =20 >=20 > =E2=80=98guix publish -C=E2=80=99 always compresses store items, unless t= hey are regular > files ending in =E2=80=9C.gz=E2=80=9D, =E2=80=9C.xz=E2=80=9D, etc. So th= ere=E2=80=99s no problem here. I see, ok. >=20 > It seems to me that the only benefit of compressing checkouts would be > the use case you=E2=80=99re talking about. For everyday use, it=E2=80=99= s actually more > efficient to leave the uncompressed, so that people don=E2=80=99t end up = doing > round of compression/decompression when they run =E2=80=9Cguix build=E2= =80=9D. Often, people would end up fetching the compressed source from a substitute server, no? And currently I already need to uncompress what I get from a substitute server before building. But now I'm wondering why we bother with the "repack" portion of "patch-and-repack"? > That said, at worst, this could be addressed by writing a specific tool > for the job. Yes. I was hoping it could be avoided though. In the meantime I can continue to monitor the situation. `~Eric