From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Enge Subject: Re: Parallel (de-)compression Date: Sun, 6 Dec 2015 16:31:42 +0100 Message-ID: <20151206153142.GC4969@debian> References: <20151202184550.GA5741@debian> <87twnyi889.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:47647) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a5bI1-0005fq-9I for guix-devel@gnu.org; Sun, 06 Dec 2015 10:31:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a5bI0-0006g0-Dm for guix-devel@gnu.org; Sun, 06 Dec 2015 10:31:53 -0500 Content-Disposition: inline In-Reply-To: <87twnyi889.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Ludovic =?iso-8859-15?Q?Court=E8s?= Cc: guix-devel@gnu.org On Fri, Dec 04, 2015 at 03:44:38PM +0100, Ludovic Courtès wrote: > Are you saying that xz-compressing TeX Live to resend it to > hydra.gnu.org is too CPU-intensive? That depends on your definition of "too". In any case, on the Novena board with an SSD attached, CPU is the limiting factor during this phase, pushing the CPU load on one core to 100% (while the other cores are idle). > Another option would be to trade compression ratio for reduced CPU usage > by using, say, ‘xz -2’ or ‘gzip’. > We did something similar in 5ef9d7d to reduce CPU consumption on the > front-end. Usually it’s much less important to reduce CPU consumption > on the build machines, but your experience seems to suggest otherwise. If possible, it would be more interesting to leverage the several cores and not make sacrifices on the compression quality. Note that I also did not measure the different timings: Compression is, I think, done separately from sending the compressed file; then it is entirely possible that the data transfer takes longer than the compression. Andreas