From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ricardo Wurmus Subject: Speeding up =?utf-8?B?4oCcZ3VpeCBwdWxs4oCdOg==?= splitting modules Date: Sun, 05 Jan 2020 21:37:36 +0100 Message-ID: <87k1657i7j.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:55859) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ioCek-0004YZ-Im for guix-devel@gnu.org; Sun, 05 Jan 2020 15:37:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ioCej-0008Jh-DK for guix-devel@gnu.org; Sun, 05 Jan 2020 15:37:50 -0500 List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org Hi Guix, thanks to Ludo=E2=80=99s excellent work on =E2=80=9Cguix pull=E2=80=9D, upd= ating Guix has become a lot faster than in the old days. With build farm support one usually only needs to wait a little while to be able to download all parts of an up-to-date Guix. Unfortunately, changes to the repository invalidate expensive derivations regularly, and waiting for the build farm to get around to building these derivations gets old quickly. Running =E2=80=9Cguix pull=E2= =80=9D repeatedly is a drag because the chances of getting a fully remotely built Guix are pretty slim. Part of the problem is that there=E2=80=99s just one derivation for the compilation of all package modules. This one derivation=E2=80=99s output i= s an input to another expensive derivation: the one for the Guix System (services, etc). We should aim to split up the guix-packages derivation, so that the guix-system derivation will not have to be rebuilt when its packages aren=E2=80=99t affected. Unfortunately, it=E2=80=99s difficult to build mo= dules independently from one another because of the many inter-module dependency cycles. G=C3=A1bor once suggested an iterative approach of identifying the most important nodes in the package graph that should be moved to their own modules, so that we would end up with a package graph that looks less like a hair ball. I think it would useful to get a better view on the relationships between modules. On the other hand: this would need to be an ongoing effort. Newly introduced packages or even new features might create complex module cycles. It sounds tedious to keep track of this and to enforce boundaries. Is there an alternative? Could we, for example, make all lookups of cross-module package references lazy? Or could we improve compilation times by disabling optimizations? Or by marking the modules as declarative and thus unlock more optimizations (with Guile 3)? -- Ricardo