From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id OxQbFi7klF/HVwAA0tVLHw (envelope-from ) for ; Sun, 25 Oct 2020 02:34:22 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id aD5rES7klF+sLwAAbx9fmQ (envelope-from ) for ; Sun, 25 Oct 2020 02:34:22 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 919EC940149 for ; Sun, 25 Oct 2020 02:34:21 +0000 (UTC) Received: from localhost ([::1]:54700 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWVrP-00062l-BB for larch@yhetil.org; Sat, 24 Oct 2020 22:34:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49128) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWVrA-00062d-EZ for bug-guix@gnu.org; Sat, 24 Oct 2020 22:34:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:52248) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kWVr8-0007Mw-22 for bug-guix@gnu.org; Sat, 24 Oct 2020 22:34:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kWVr7-0001sK-VH for bug-guix@gnu.org; Sat, 24 Oct 2020 22:34:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#44175: [optimization] Grafting is too slow Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 25 Oct 2020 02:34:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44175 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 44175-submit@debbugs.gnu.org id=B44175.16035931977152 (code B ref 44175); Sun, 25 Oct 2020 02:34:01 +0000 Received: (at 44175) by debbugs.gnu.org; 25 Oct 2020 02:33:17 +0000 Received: from localhost ([127.0.0.1]:35561 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWVqP-0001rI-AM for submit@debbugs.gnu.org; Sat, 24 Oct 2020 22:33:17 -0400 Received: from mail-qt1-f179.google.com ([209.85.160.179]:35696) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWVqN-0001r4-GI for 44175@debbugs.gnu.org; Sat, 24 Oct 2020 22:33:16 -0400 Received: by mail-qt1-f179.google.com with SMTP id c15so4380015qtc.2 for <44175@debbugs.gnu.org>; Sat, 24 Oct 2020 19:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=O67LJmwuJroGsWbSJUZdoqLA+n2Tp4+T1z0Qt+f0Qio=; b=LaAnhgoycQHd4E4BXwO7ZizQpq+beCe7p0TfBpQalOSjPhpEJjfpQH+zj4ssURAfjK EHHH7PRkKvMybhVeOyJcTIRQfIs4xC/CzFGEI70OpEKY9flpqmNTRWMKrciW/wqdzybN IMv6E0wbESZsC0znyaMM62miqSDbDlypiRgKtIM19+RLz9cbbxp2GRG4+ZP4G/+eJev0 qedvGKun4PGC1KrOvoObSebmu6SHNahY3CUD0nDT/JhkTUlL6/mmyNc/8/+8/WamhBmV 6vKXDOWMi4bc3HqZGsUG4MrQEGSnNfSgh4DajKg3FReb4oz9hVw+Co2sQPkemfDLwYpc FOqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=O67LJmwuJroGsWbSJUZdoqLA+n2Tp4+T1z0Qt+f0Qio=; b=TkIsD2TGCUDg1pEsF/aoPTpQ9+fqm4Bm1V6pi2RmH8eVLJvHttkxgqpedUD8yXnZiX VhaRWxt7Rs4cOKvA8PZnmuFpVex5q/4YZ76fulQBjX0l+g9KLWK8p67YpPlIyIvPZUuQ 2RXxwZH1OjE9isDi0vOyxr4rYMh3HTsupmQ03zOm6UvPxCmsyDBpc2v/aFGMBE5AysKI bALMBbeqgdv7VyDngwUz9NG5Adv2A7mMEKk15A8Y/K6ifUfDAmN9A6jMzLnB/Ri7xgTe VWhzCs2+ljvOZtYjX333TcZoElhjwN4ea/1ysmGDuRC6v+kDdR9lh+fFquyw9wo5jmTw 7iZQ== X-Gm-Message-State: AOAM532zEhYXcJTXNI3vHtCMyuCx5hras9R8ZTrzVcTEzjpth2Im0itN 5OaKE6zxlP+yGvmrfZxeWIAQ/aD45BA= X-Google-Smtp-Source: ABdhPJwkp/6SqD6i3nMbM2t1Z7uesNsvpwnX+rvhDS/oTCuctfW3fm2C8R+6Zp7alZ1/FjNrlG8UVw== X-Received: by 2002:ac8:64d:: with SMTP id e13mr10372093qth.23.1603593189822; Sat, 24 Oct 2020 19:33:09 -0700 (PDT) Received: from hurd (dsl-159-173.b2b2c.ca. [66.158.159.173]) by smtp.gmail.com with ESMTPSA id t70sm3879298qke.119.2020.10.24.19.33.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 24 Oct 2020 19:33:09 -0700 (PDT) From: Maxim Cournoyer References: <87y2o3zf00.fsf@gnu.org> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@gnu.org> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@gnu.org> <20200706125800.GA3176@zpidnp36> <20200720095059.GB3143@zpidnp36> <87mu3tq2hh.fsf@gnu.org> <87r1pqc6sy.fsf@gmail.com> <20201023072621.GA3115@zpidnp36> <875z71ccq0.fsf_-_@gmail.com> <87lffwk5fm.fsf@gnu.org> Date: Sat, 24 Oct 2020 22:33:08 -0400 In-Reply-To: <87lffwk5fm.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Fri, 23 Oct 2020 23:17:49 +0200") Message-ID: <87o8kr2fx7.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 44175@debbugs.gnu.org, Lars-Dominik Braun Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=LaAnhgoy; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: 8AHYJnB+mYeX Hello! Ludovic Court=C3=A8s writes: > Maxim Cournoyer skribis: > >> Lars-Dominik Braun writes: >> >>> Hi Maxim, >>> >>>> Judging from the above, it seems this issue has been resolved. >>> grafting is still a performance issue imo. Compare for example: >>> >>> $ time guix environment --ad-hoc --search-paths r-learnr >>> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s sys= tem 210% cpu 2,844 total >>> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts >>> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s us= er 0,08s system 164% cpu 1,277 total >> >> I'm opening a new issue to track optimizing the grafting code, since >> it's independent of environments (grafts are applied anytime a >> derivation is built, AFAICT). Grafting is inherently IO-bound, > > What is slow above is not grafting itself: it=E2=80=99s determining what = to > graft that takes CPU time. On my system, grafting seems IO rather than CPU bound, I'm guessing because of the need to scan all the files for strings to replace in the graft process. > I had reopened the initial bug at ; > should we close this one? Many optimizations were made in the above issue that were not related to the grafting process, so to me a fresh entry such as this one is clearer to follow. That said, feel free to proceed as you see fit, being the issue "owner" :-). Thanks, Maxim