From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id 8LJGETGvI2c5bAAAqHPOHw:P1 (envelope-from ) for ; Thu, 31 Oct 2024 16:24:17 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id 8LJGETGvI2c5bAAAqHPOHw (envelope-from ) for ; Thu, 31 Oct 2024 17:24:17 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=K1edkuXI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1730391857; a=rsa-sha256; cv=none; b=eCZ1OvGzqT4dlaSgl/aZPuVPwleHtTvfoafhxYP0g/KeaMl+7JR5JnnUjU1ZOV8OYxZmb2 lqaRv0DRqUn76VDy837Ggvzrl6xBUmFQmP+6tfAQf280cYDjjo84/UkXN9rIi4qq+bfGhP /3Gcpuui04qHaW3znDB+Nx9KombGlD4fihPYVyMXmztEgfyexTKQ/AcTIdDTr7c67CkY8g dxf95nari8BKzqYmROCmIWwb14Q/luZ9Hti4ubUv6eECY7eAr49bUAGCX0vGR5xmX7r7VR 0seQ8AaxoKoI+FvkiRTqadZNMgA0tqSCqvAxPtBe/LBQQCR7eAMryNBKO8C2+Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=K1edkuXI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1730391857; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=9eDldlo03feOqSszO+WEUqPbNjjj+XsANcpCO9bCDFE=; b=AUxbhZy+JN6T7zzJIHQF1eJuVKI8iSgdoSRRpr5FPWuB0G2thTdNOeilZxr6jDjiur6hwp xewIpNYB9cgodvG4ac3y+8HA3JNoAakQe8sNhyop4mGGOSfSUHanFd1n1dSynTvZLTI3X8 A30L70CwOV5DTFLukC9y8PfXWenVok1UWUQEuOp4/MzeEtaK6fk5/r9cPfu1tbdk6XTE0H gWEnGUyXLq80mSKzMhi2kD2kt7RtaAo+/wXic4f3E2nUciskmC260mibiqghJs7iF69uPv rmyJaCr23sAbFe038FA6cwzcgVE+S5/UlQd5J5OzBA7dlQNVFcB8JlnNs6rCeg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 07EAE8ABE3 for ; Thu, 31 Oct 2024 17:24:17 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6Xxm-00007w-Lg; Thu, 31 Oct 2024 12:23:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6Xxk-000053-Kd for guix-devel@gnu.org; Thu, 31 Oct 2024 12:23:56 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t6Xxj-00022m-2K for guix-devel@gnu.org; Thu, 31 Oct 2024 12:23:56 -0400 Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2e2fb304e7dso877147a91.1 for ; Thu, 31 Oct 2024 09:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730391833; x=1730996633; darn=gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=9eDldlo03feOqSszO+WEUqPbNjjj+XsANcpCO9bCDFE=; b=K1edkuXIsr4ZaW2RNLes0byEwKtnlrsI440qnXP8bFUf3YUi68XNGbt2M6wdMH94T3 pbCMBjjynBgWpEGDYAz3Omk4bszQEBbS/mPBrEE4f0OwPN6RzJI1lpivOTUcC++2Ptas tdw1iefHi0PUozWLN9HJq5ndTrzkvzJnDBcWAxr25j4tXSE3Hu6ENOOQ2dSZpIbOO7Ba EXbk9nmkiaTIANyMlqjD0J1hmsl1R3NBJTNJLS0e04g1ILbjXYGKPQEmgBJbJ9p6CtWg KBe1GoW3YhEgIpMvwJPbdZAB3zdLJHk3QaKB2CsJoyYICDkNvVwR8YXDqNkxM7zNXoVJ Xasw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730391833; x=1730996633; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9eDldlo03feOqSszO+WEUqPbNjjj+XsANcpCO9bCDFE=; b=q+XRPNI1ls6nzbASFPKqBC4XmWRuYrnVlUqX0miIUgMv+hXx93vHzFjIxG6Ru17FF/ On+XAfF5Kc6aGWtxfnxROsHMJztnvfeDeTkb49/+Dd08uNG7txoWZUixyZ4ZjSHgMDQU ijYnK+6u87YeAgxgApGm+pnd1MUfN0biwYbonSqV9oLQYB7xZJ951fq1Jp8Jx2FG1W5y aQYkGUK+FR3Zs4RPXpcUIOEL8NRCpJ6HlHNfP055pgLypEYZW1wFN6HYtNuGucnXQ5cR dEl/+9zEdJtDOzxp7lk5/o/rO9d41CeK1nrouGHj2ppx4QgH3jINlC49wzpds6VEUWR5 xmEQ== X-Gm-Message-State: AOJu0YyIWjDLvRH6hYiDMQxev+96aJXfzgqlen8D6KBd5ig5utMe1bXE WZlj7yRpD7QXTtDmVtcffXwPY/tvMV/8v2nWTZy8VSL48h6rAuyIwJV/Bd63MQmVCRkXAzvqXYV dTDA9OG5JxEbgfvYZd85G3TPc3cQWMw== X-Google-Smtp-Source: AGHT+IHMrXFXqamPP+2MBXsBzql+FSpJN/dZsFIvkTutkqKSX52Zq0VpvkQMjG0KrQs5kOLasvunShc2oPIRlAOqbBg= X-Received: by 2002:a17:90b:1850:b0:2e2:8995:dd10 with SMTP id 98e67ed59e1d1-2e8f0f53f85mr22336075a91.4.1730391833413; Thu, 31 Oct 2024 09:23:53 -0700 (PDT) MIME-Version: 1.0 References: <87ldy4e4ub.fsf@gmail.com> <877c9o6yfi.fsf@whispers-vpn.org> In-Reply-To: <877c9o6yfi.fsf@whispers-vpn.org> From: Evan Cooney Date: Thu, 31 Oct 2024 11:23:41 -0500 Message-ID: Subject: Re: Including code in a non-Guile language into Guix To: Runciter Cc: guix-devel@gnu.org Content-Type: multipart/alternative; boundary="0000000000008d8ff40625c83eb4" Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=evancooney71@gmail.com; helo=mail-pj1-x102b.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -1.50 X-Spam-Score: -1.50 X-Migadu-Queue-Id: 07EAE8ABE3 X-Migadu-Scanner: mx12.migadu.com X-TUID: aLEdZQ5L/Kt6 --0000000000008d8ff40625c83eb4 Content-Type: text/plain; charset="UTF-8" Would it be useful for me to wrote a script to collect performance data for various stages? Like downloading, computing derivations, building profile etc? On Thu, Oct 31, 2024, 11:18 AM Runciter via Development of GNU Guix and the GNU System distribution. wrote: > Suhail Singh writes: > > > Daniel Littlewood writes: > > > >> guix pull ("38k new commits"): 21m45s > >> guix pull immediately after: 2m25s > >> guix shell emacs (fresh): 1m49s > >> ... > >> > >> nix-channel --update: 0m23s > >> nix shell -p emacs (fresh): 0m24s > > > > Those are some interesting comparisons. Is the reason guix pull takes > > so long as compared to updating nix-channel primarily due to the > > authentication of commits? Or something else? > > As far as the local machine computations go, clearly, authenticating the > commits is not the bottleneck. On all machines, indexing the received > git objects locally is much longer than authenticating the commits. > > On my X60, when I pull for the first time after I delete the cache, the > indexing step alone takes more than 40 minutes. > > The 2m25s that Daniel had for his second git pull, that had to be spent > mostly on computing the Guix derivation. This time is in large part > in-compressible I guess. Not that I know of a lot about this, but by > reading the output, it's clear that every time guix pull has to compute > the whole derivation of the latest commit of all the > channels. Apparently, in a pull where Guix determines that it has > nothing to do, this step is required before Guix can make the > determination that it has nothing to do... > > > --0000000000008d8ff40625c83eb4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Would it be useful for me to wrote a script to collect perfo= rmance data for various stages? Like downloading, computing derivations, bu= ilding profile etc?


On Thu= , Oct 31, 2024, 11:18 AM Runciter via Development of GNU Guix and the GNU S= ystem distribution. <guix-devel@gn= u.org> wrote:
Suhail Singh &= lt;suhailsingh247@gmail.com> writes:

> Daniel Littlewood <danielittlewood@gmail.com> writes:=
>
>> guix pull ("38k new commits"): 21m45s
>> guix pull immediately after: 2m25s
>> guix shell emacs (fresh): 1m49s
>> ...
>>
>> nix-channel --update: 0m23s
>> nix shell -p emacs (fresh): 0m24s
>
> Those are some interesting comparisons.=C2=A0 Is the reason guix pull = takes
> so long as compared to updating nix-channel primarily due to the
> authentication of commits?=C2=A0 Or something else?

As far as the local machine computations go, clearly, authenticating the commits is not the bottleneck. On all machines, indexing the received
git objects locally is much longer than authenticating the commits.

On my X60, when I pull for the first time after I delete the cache, the
indexing step alone takes more than 40 minutes.

The 2m25s that Daniel had for his second git pull, that had to be spent
mostly on computing the Guix derivation. This time is in large part
in-compressible I guess. Not that I know of a lot about this, but by
reading the output, it's clear that every time guix pull has to compute=
the whole derivation of the latest commit of all the
channels. Apparently, in a pull where Guix determines that it has
nothing to do, this step is required before Guix can make the
determination that it has nothing to do...


--0000000000008d8ff40625c83eb4--