From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wjX0LFqrlV/jYAAA0tVLHw (envelope-from ) for ; Sun, 25 Oct 2020 16:44:10 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id +M8tKFqrlV+UdwAAB5/wlQ (envelope-from ) for ; Sun, 25 Oct 2020 16:44:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3695A94043A for ; Sun, 25 Oct 2020 16:44:10 +0000 (UTC) Received: from localhost ([::1]:59602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWj7p-0000Yj-1K for larch@yhetil.org; Sun, 25 Oct 2020 12:44:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWj7i-0000Yc-Hu for bug-guix@gnu.org; Sun, 25 Oct 2020 12:44:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54171) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kWj7i-0003mc-8O for bug-guix@gnu.org; Sun, 25 Oct 2020 12:44:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kWj7i-0004gM-6M for bug-guix@gnu.org; Sun, 25 Oct 2020 12:44:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#44175: [optimization] Grafting is too slow Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 25 Oct 2020 16:44:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44175 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Maxim Cournoyer Received: via spool by 44175-submit@debbugs.gnu.org id=B44175.160364420317944 (code B ref 44175); Sun, 25 Oct 2020 16:44:02 +0000 Received: (at 44175) by debbugs.gnu.org; 25 Oct 2020 16:43:23 +0000 Received: from localhost ([127.0.0.1]:37484 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWj75-0004fL-8u for submit@debbugs.gnu.org; Sun, 25 Oct 2020 12:43:23 -0400 Received: from eggs.gnu.org ([209.51.188.92]:60102) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWj73-0004f7-59 for 44175@debbugs.gnu.org; Sun, 25 Oct 2020 12:43:21 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37194) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWj6x-0003hk-Mu; Sun, 25 Oct 2020 12:43:15 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=57270 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kWj6w-0006km-M4; Sun, 25 Oct 2020 12:43:15 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87y2o3zf00.fsf@gnu.org> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@gnu.org> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@gnu.org> <20200706125800.GA3176@zpidnp36> <20200720095059.GB3143@zpidnp36> <87mu3tq2hh.fsf@gnu.org> <87r1pqc6sy.fsf@gmail.com> <20201023072621.GA3115@zpidnp36> <875z71ccq0.fsf_-_@gmail.com> <87lffwk5fm.fsf@gnu.org> <87o8kr2fx7.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Brumaire an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 25 Oct 2020 17:43:12 +0100 In-Reply-To: <87o8kr2fx7.fsf@gmail.com> (Maxim Cournoyer's message of "Sat, 24 Oct 2020 22:33:08 -0400") Message-ID: <87wnzedzof.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 44175@debbugs.gnu.org, Lars-Dominik Braun Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.51 X-TUID: NjMBRaGPl5by Hi! Maxim Cournoyer skribis: >>>> $ time guix environment --ad-hoc --search-paths r-learnr >>>> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s sy= stem 210% cpu 2,844 total >>>> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts >>>> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s u= ser 0,08s system 164% cpu 1,277 total >>> >>> I'm opening a new issue to track optimizing the grafting code, since >>> it's independent of environments (grafts are applied anytime a >>> derivation is built, AFAICT). Grafting is inherently IO-bound, >> >> What is slow above is not grafting itself: it=E2=80=99s determining what= to >> graft that takes CPU time. > > On my system, grafting seems IO rather than CPU bound, I'm guessing > because of the need to scan all the files for strings to replace in the > graft process. Yes, you=E2=80=99re right of course. But I think in the example above Lars= was reporting the run time of the =E2=80=98guix=E2=80=99 command when the graft= itself is already built. Just the extra code to compute the graft derivation (not actually building them) leads to x2 wall-clock time or so. >> I had reopened the initial bug at ; >> should we close this one? > > Many optimizations were made in the above issue that were not related to > the grafting process, so to me a fresh entry such as this one is clearer > to follow. That said, feel free to proceed as you see fit, being the > issue "owner" :-). Alright, thanks! Ludo=E2=80=99.