From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id SEUxALzVkl9lDwAA0tVLHw (envelope-from ) for ; Fri, 23 Oct 2020 13:08:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id mMhFNrvVkl9hcQAAB5/wlQ (envelope-from ) for ; Fri, 23 Oct 2020 13:08:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 11B7B9402A8 for ; Fri, 23 Oct 2020 13:08:11 +0000 (UTC) Received: from localhost ([::1]:56122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kVwnh-0003ZY-E8 for larch@yhetil.org; Fri, 23 Oct 2020 09:08:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50468) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kVwna-0003ZC-GQ for bug-guix@gnu.org; Fri, 23 Oct 2020 09:08:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:44763) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kVwna-0007jw-6w for bug-guix@gnu.org; Fri, 23 Oct 2020 09:08:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kVwna-0003aK-30 for bug-guix@gnu.org; Fri, 23 Oct 2020 09:08:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#44175: [optimization] Grafting is too slow Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 23 Oct 2020 13:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 44175 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Lars-Dominik Braun X-Debbugs-Original-Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , bug-guix Received: via spool by submit@debbugs.gnu.org id=B.160345846413755 (code B ref -1); Fri, 23 Oct 2020 13:08:01 +0000 Received: (at submit) by debbugs.gnu.org; 23 Oct 2020 13:07:44 +0000 Received: from localhost ([127.0.0.1]:56309 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kVwnI-0003Zn-Dn for submit@debbugs.gnu.org; Fri, 23 Oct 2020 09:07:44 -0400 Received: from lists.gnu.org ([209.51.188.17]:44862) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kVwnG-0003Zf-Kd for submit@debbugs.gnu.org; Fri, 23 Oct 2020 09:07:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50314) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kVwnG-0003XY-Fu for bug-guix@gnu.org; Fri, 23 Oct 2020 09:07:42 -0400 Received: from mail-qv1-xf32.google.com ([2607:f8b0:4864:20::f32]:44970) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kVwnE-0007gH-QA; Fri, 23 Oct 2020 09:07:42 -0400 Received: by mail-qv1-xf32.google.com with SMTP id s17so605256qvr.11; Fri, 23 Oct 2020 06:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=HNPJPbwHB+tUEYnv62k4WakSz/x7WlEkRe9wVFNPQOg=; b=NQcw0p7LC5EBS9YpJDk3OTy9qvha4NTkjgKTeIufc7s4pv2tazrI6fcE7sUnFbz33p n9ld+SOHYOjeCSYyQFjWU5UN8Bsb4Vngy/jGSicdmlUWYMJi29hDqQgsnvGB76j0+3XB tpELpFpH+A2dT6CamXytuDo1eVwjTh6Djw1uaGWPSHi3kBD/Jo+olIpDZ3nS6KDFL6WU ScgI8CtngqjqgpraKEjxq2ShPNz/jAz4WGDk3gUOiuibRuS+pjJ8TnWcRyeYEF8tlOr6 RZoqSs9ocKYNj+WqY/6WHlWcUUaAK1wihparRVlAvN0jd8GWIbzior3XQbFrm06Ib8fH KeNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=HNPJPbwHB+tUEYnv62k4WakSz/x7WlEkRe9wVFNPQOg=; b=UEziSPDrCtjJqgyPlM+MYAXTpuwakgiazb+BwazvoNyVEujWylGbPjcGhEAEgj53c4 RZUyWK+EFX/N4qCyuKgJYDtwzQQVB9Z2Ni0Ydiu48YV2V+elN1oLM388nm4tenQ+nRdU vs3EeRVG6UaBgxFi4m2IXXICKQvcsnh3vpELNoSqPvW4CDVfbu/h+fOPDyySLf7TGs0n 4dXj1iBxQSlxWj/UUbJ/K3Mrjlt8qGlKcGETXi4JVgXin8lz7eNy7x41ZzO47ZW1lIEG 2gfXuNM4A9Gw4KqRpd6cZLkgupPqt22Du+4caNxr3BTvGPT/QeBIxfqOfmv/e4v43R2+ 387Q== X-Gm-Message-State: AOAM531colYN4pQ4Je5fxyPnLPK42zXjG18b69raW7S0GTvhaYIrHaxk F7TmD4plh5kAUyQFr9mfbL4W61QOgw63Cw== X-Google-Smtp-Source: ABdhPJyuCJqg7n6R0sKqhfg+IGr5o6A8jHacZm029jDCm1PFzaYviqxb+B7fUh5fve+ZqRrk3ggz8w== X-Received: by 2002:a0c:c985:: with SMTP id b5mr2385243qvk.10.1603458457352; Fri, 23 Oct 2020 06:07:37 -0700 (PDT) Received: from hurd (dsl-159-173.b2b2c.ca. [66.158.159.173]) by smtp.gmail.com with ESMTPSA id u2sm770111qtw.40.2020.10.23.06.07.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 06:07:36 -0700 (PDT) From: Maxim Cournoyer References: <87y2o3zf00.fsf@gnu.org> <20200701112453.GB25179@zpidnp36> <87eepuyk99.fsf@gnu.org> <20200702070000.GA4532@zpidnp36> <20200702120333.GB4532@zpidnp36> <87k0zhkoyf.fsf@gnu.org> <20200706125800.GA3176@zpidnp36> <20200720095059.GB3143@zpidnp36> <87mu3tq2hh.fsf@gnu.org> <87r1pqc6sy.fsf@gmail.com> <20201023072621.GA3115@zpidnp36> Date: Fri, 23 Oct 2020 09:07:35 -0400 In-Reply-To: <20201023072621.GA3115@zpidnp36> (Lars-Dominik Braun's message of "Fri, 23 Oct 2020 09:26:21 +0200") Message-ID: <875z71ccq0.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::f32; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf32.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -2.3 (--) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 44175@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=NQcw0p7L; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: AlKfMGctFQZv Hello Lars, Lars-Dominik Braun writes: > Hi Maxim, > >> Judging from the above, it seems this issue has been resolved. > grafting is still a performance issue imo. Compare for example: > > $ time guix environment --ad-hoc --search-paths r-learnr > guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s system 210% cpu 2,844 total > $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts > guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user 0,08s system 164% cpu 1,277 total I'm opening a new issue to track optimizing the grafting code, since it's independent of environments (grafts are applied anytime a derivation is built, AFAICT). Grafting is inherently IO-bound, since it must scan all the built artifacts of a grafted package (from its dependency that was 'replaced' all the way up), if I understood correctly. "info (guix) Security Updates" gives some information. Perhaps we could benchmark how fast our code currently is for grafting, and compare it with the fastest sed-like utility out there, as a starting point, to have an idea of how much there is to gain from optimization. Maxim