From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qFx/Jk0lnmGEmwAAgWs5BA (envelope-from ) for ; Wed, 24 Nov 2021 12:43:09 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id kBc7Ik0lnmGsWgAAB5/wlQ (envelope-from ) for ; Wed, 24 Nov 2021 11:43:09 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3304B2037B for ; Wed, 24 Nov 2021 12:43:09 +0100 (CET) Received: from localhost ([::1]:38462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mpqg7-00053J-N5 for larch@yhetil.org; Wed, 24 Nov 2021 06:43:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44704) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mpqfa-00050o-I1 for guix-devel@gnu.org; Wed, 24 Nov 2021 06:42:34 -0500 Received: from [2a00:1450:4864:20::329] (port=52986 helo=mail-wm1-x329.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mpqfY-0002eI-NI for guix-devel@gnu.org; Wed, 24 Nov 2021 06:42:34 -0500 Received: by mail-wm1-x329.google.com with SMTP id o29so2030357wms.2 for ; Wed, 24 Nov 2021 03:42:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=6wyOKlTFvPbiOAmqs+re3amhOwqXuu/84zUcvqP6pFY=; b=piRA5qK0fUCXpqcKIRe4bVBL0jhwhgrK7DHJGy0v/m6aM/p7xns4ZJgNs6wlBuBk/g sNsx7dFdE2yWNfCKWtNQr2yhHdPSk5t8JbIA9s459pSbSbLz+YYziJyf0oKiw2AG5L1W w2yWn9QBz3OWQoR4cEAvQrBteQsNjjDdt7N31Clwkr4PHnhz2kbEAZ/2sDAGxkk71s0Z CG0Z0TD9JJJTkp4GdGIBGmfq5vdAhKMyAFvZS7bQxwNIR2wob8MFqNMn/MmbA0Y5qQAa 35aTl3ErksuafAoJYBxmlGAYv3X4wUX48VCVBupE2vdv65cnzYvRD3LBWL+IBtk5a4J3 +43Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=6wyOKlTFvPbiOAmqs+re3amhOwqXuu/84zUcvqP6pFY=; b=AD518vhK/66uTHFerMTDPtsLU9mjVB0YP7jVssH1chXPX+6r4AzlblcaIQym5jvp2T CkD4SiJXxaodKA2r1YIWxcBf7CBDx9BjLr1hRUdp/wSl/j9iaJsEqFUnu/IAQ6yTl66O sWI2TuFK5bqEcS8x1+pREMzjnF5zDu4c0nJVxhbO7Wltks2w0i49eBVBWmeJL52gtPg0 b8iVcw35vcnI+DZD90yp9KOtELVlKktAoPw2ypfYYGwfqaGA5TlXEPe7yPueYV7mY3st xIwH8e02s0BxvCWMoyBlUCKJa+pNF/Hi+zAE4otWsp1xwaUumkvBpYMFSIY7PrYMLiZL gUvQ== X-Gm-Message-State: AOAM533AdKluzLjQwtK9hfIvPMefuvGA2dOaC/4u+P/AKvJ9qCssIZyC ixtM2mkF8mXqtfcoIVXO7PIsB1ORt9k= X-Google-Smtp-Source: ABdhPJznoDohFLow4FaBaTXV1AQO12NiOTKceHtkpTJbMxIu85ZpZoE/68F4xiFzqberitb08zSNPA== X-Received: by 2002:a7b:cd96:: with SMTP id y22mr13948306wmj.121.1637754151240; Wed, 24 Nov 2021 03:42:31 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id n32sm4852590wms.42.2021.11.24.03.42.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Nov 2021 03:42:30 -0800 (PST) From: zimoun To: Jacob Hrbek , Julien Lepiller Subject: Re: Proposal: Build timers In-Reply-To: References: <868rxfwuib.fsf@gmail.com> <865ysjw0ek.fsf@gmail.com> Date: Wed, 24 Nov 2021 12:35:33 +0100 Message-ID: <86r1b5u6pm.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::329 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=zimon.toutoune@gmail.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "guix-devel@gnu.org" Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1637754189; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=6wyOKlTFvPbiOAmqs+re3amhOwqXuu/84zUcvqP6pFY=; b=o8OOUswzESYZSufN/mUwiOlxv5vxP8npjWR9PfhDh7U3r6FozVztDg0yghmNb/IXmPrpAq aRGIvAnoGb57ij3eM1dVfZJ9BDYCix3ZoC2u04piJYSztpN6mMq04zZAx/YdYP+izhP9ku RA3aqbM2I645D03+7KnJovGNKyAsdfn/Y+JhhVU5OURafWyUI/Ez8dz+pKQzjEhMWXgqXm /gALCu3XI7LzOqnb/5nY9CmOdRY+NaDYuDwGBoXptnhFp3UL9Z2WeRqYMX+P0H3JWaZd+3 Z1GyeQyAQIqtirQJBaFHeaTuY0KRrVs5CeSTqgjJccSrQhC1YAlxNIp35i1phA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1637754189; a=rsa-sha256; cv=none; b=VVJuAgyRsuMl8FY4PuL+/36bIN60oLhSl3Z4lWmBQVjxd6MyrTzORUDUykmprZpgNADgwn L89RCiIsN80fgDROgq1JZtdidGJjmnkoorQKzlNYo/NKhnVQBorxfNFiNaZb3zdaoNbeY3 7E//P0ic8C1k5hP7SFFnTHyfQI5QYAa/DvILR3LSoAbsk+6B5Bjw2HcTpcxPBFjc9ghQ+1 WqeTB6VwUQ3CL+wF9Dgg6kT3zYuex6qv74GdjwGEo++MvQm0zdP4lSrpveoKCmgGYDlv+q 252pD9TWlyu771bcYjRF55RWDSabkQETA6uvaPd5ea8WHiHf3SkcDgLAU0+Bdw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=piRA5qK0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.09 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=piRA5qK0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3304B2037B X-Spam-Score: -4.09 X-Migadu-Scanner: scn0.migadu.com X-TUID: i7R1OaSYusi2 Hi, On Tue, 23 Nov 2021 at 14:39, Jacob Hrbek wrote: >> This approximation would not even be accurate enough for the same >> machine. For instance, the test suite of the julia package runs >> mainly sequential using one thread...=20 > > I am aware of this scenario and I adapted the equasion for it, but I > recognize that this exponentially increases the inaccuracy with more > threads and I don't believe that there is a mathematical way with the > provided values to handle that scenario so we would have to adjust the > calculation for those packages. What I am trying to explain is that the model cannot work to be predictable enough with what I consider a meaningful accuracy. Obviously, relaxing the precision, it is easy to infer a rule of thumb; a simple cross-multiplication fits the job. ;-) The =E2=80=9Cpok=C3=A9mon-battle=E2=80=9D model is a simple linear model (cross-multiplication); using Jacob=E2=80=99s =E2=80=9Cnotation=E2=80=9D: - HP: time to build on machine A - DPS =3D nthread * cpufreq : =E2=80=9Cpower=E2=80=9D of machine Then it is expected to evaluate =E2=80=99a=E2=80=99 and =E2=80=99b=E2=80=99= on average such that: HP =3D a * DPS + b based on some experiments. Last, on machine B, knowing both nthread' and cpufreq' for that machine B, you are expecting to evaluate HP' for that machine B applying the formula: HP' =3D a * nthread' * cpufreq' + b Jacob, do I correctly understand the model? In any case, that=E2=80=99s what LFS is doing, instead HP is named SBU. And instead DPS, they use a reference package. And this normalization is better, IMHO. Other said, for one specific package considered as reference, they compute HP1 (resp. HP2) for machine A (resp. B), then for machine A, they know HP for another package and they deduce, HP' =3D HP2/HP1 * HP All this is trivial. :-) The key is the accuracy, i.e., the error between the prediction HP' and the real time. Here, the issue is that HP1 and HP2 capture for one specific package the overall time; which depends on hidden parameters as nthread, cpufreq, IO, and other parameters from hardware. But that a strong assumption when considering these hidden parameters (evaluated for one specific package) are equally the same for any other package. It is a strong assumption because the hidden parameters depends on hardware specifications (nthread, cpufreq, etc.) *and* how the package itself exploits them. Therefore, the difference between the prediction and the real time is highly variable, and thus personally I am not convince the effort is worth; for local build. That=E2=80=99s another story. ;-) LSF is well-aware of the issue and it is documented [1,2]. The root of the issue is the model based on a strong assumption; both (model and assumption) do not fit how the reality concrete works, IMHO. One straightforward way =E2=80=94 requiring some work though =E2=80=93 for = improving the accuracy is to use statistical regressions. We cannot do really better to capture the hardware specification =E2=80=93 noticing that the machine s= tress (what the machine is currently doing when the build happens) introduces a variability hard to estimate beforehand. However, it is possible to do better when dealing with packages. Other said, exploit the data from the build farms. Well, I stop here because it rings a bell: model could be discussed at length if it is never applied to concrete numbers. :-) Let keep it pragmatic! :-) Using the simple LFS model and SBU, what would be the typical error? For instance, I propose that we collectively send the timings of packages: bash, gmsh, julia, emacs, vim; or any other 5 packages for x86_64 architecture. Then we can compare typical errors between prediction and real, i.e., evaluate =E2=80=9Caccuracy=E2=80=9C for SBU and = then decide if it is acceptable or not. :-) Cheers, simon 1: 2: