From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id o317Gnqz2mJlPwAAbAwnHQ (envelope-from ) for ; Fri, 22 Jul 2022 16:26:02 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id EHtWGXqz2mIMRgAA9RJhRA (envelope-from ) for ; Fri, 22 Jul 2022 16:26:02 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3FE50F2A1 for ; Fri, 22 Jul 2022 16:26:02 +0200 (CEST) Received: from localhost ([::1]:55494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oEtbN-0006lM-FP for larch@yhetil.org; Fri, 22 Jul 2022 10:26:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56614) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oEtb9-0006l1-Se for guix-devel@gnu.org; Fri, 22 Jul 2022 10:25:47 -0400 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]:44690) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oEtb8-0002t9-8J for guix-devel@gnu.org; Fri, 22 Jul 2022 10:25:47 -0400 Received: by mail-qk1-x731.google.com with SMTP id b25so3679148qka.11 for ; Fri, 22 Jul 2022 07:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=l1I6smMnYCkA3qG7ADX5i/i9iI9YtrXDw5kTdGZU7rg=; b=fSgrX9zNH8dsSfLIX1431kKhonzrC1eayhat+sdAbHdY7ILnxKCmdoBc1uQdV/qJkn w8SRnXAglDGc64owBY9pWmzQZ7rMTe02P20WZaNVVW6yMY72DfkjrgrB8BKITeZCp7Qr buQyE5VI+v/1g++0yssFJMtpF83C62vMQPgSGtr5h7Mw5T50u7D78/tYVoO24fk4b2GF B5VOTorpXnqiasqEfm6xgvw1nZCmEDywek3F7zZaf499RWX7Y3BUcR8Qi0yLNDDcXlFi HThtbY7qLCE1JfBAKMr5q6nbMO58Cx1CM5PB4Qso6/A6DWyxXgbuE0GV1ub5mP61VAXN K1+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=l1I6smMnYCkA3qG7ADX5i/i9iI9YtrXDw5kTdGZU7rg=; b=HqoVJgjdJS83R08VOoH7LANiZ6Qe/GiS4bfo7GTZk60pBmpzQ1V/ze1JNwpr8e89th NaOTDXpbCZ6hsAncf8IcAVk6r6jB3aoZbPX2F6QHpf5QoySH68V6/t6t/6ZuQ5R9pDzv AFyWDCxSZIdCdwXcQjQf/HDlJloI4n9D6Mmp6No3HXwvs29oUqW8POZZgCykV+6Q+phF EsTAOAtc4B/kUCBbUxbkQ4Tid7QeXGQRLPf1h1atnmENB7Xor938uPX3Qp5/6E25BNiQ n51Y8d1z+b+dg54WaPZFsNZdbPw022oKBlfNod5n0X/J0yn+xPt5qRv3RknT/tvRtEuz HqUg== X-Gm-Message-State: AJIora83phM0OhX4reJV6/Duj2xf09lAiK2AdR2JmFJnD6rDSseZ3JX7 eM+i1kWEy7Ajn3MYRjDSLKMNaCu0Y8c= X-Google-Smtp-Source: AGRyM1vFifMOGBteQcTAxZfxeQo8YgGEZbCyHVbQQU4ewN7Pu9PJs4dps47e1jzgjmTNiBzXUvLgZQ== X-Received: by 2002:a37:680e:0:b0:6b6:1500:b953 with SMTP id d14-20020a37680e000000b006b61500b953mr96643qkc.19.1658499944701; Fri, 22 Jul 2022 07:25:44 -0700 (PDT) Received: from washu-v4 (172-221-246-205.res.spectrum.com. [172.221.246.205]) by smtp.gmail.com with ESMTPSA id f5-20020ac859c5000000b0031eef540614sm3160228qtf.51.2022.07.22.07.25.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jul 2022 07:25:44 -0700 (PDT) From: Katherine Cox-Buday To: jgart Cc: Guix Devel Subject: Re: Is Guix suitable for large monorepos? References: <20220721235528.GB10719@gac> Date: Fri, 22 Jul 2022 09:25:43 -0500 In-Reply-To: <20220721235528.GB10719@gac> (jgart@dismail.de's message of "Thu, 21 Jul 2022 23:55:28 -0500") Message-ID: <87zgh1teag.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::731; envelope-from=cox.katherine.e@gmail.com; helo=mail-qk1-x731.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1658499962; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=l1I6smMnYCkA3qG7ADX5i/i9iI9YtrXDw5kTdGZU7rg=; b=f+nVFdl1bRBwgO0gLiPZeY1RFTAz01KYSm4x5AUaox/5fv6cbTTKJ02lty1+udmH4TNVRq oJxWKlKN3CTJB156HT4su+5p/69irhj8/aAAJb0oEj6VmWvGM4jz5D+o0iBa6AgvaKjfv5 3oBX7CoUchnKcAbMrN5GVlX6q2g+BnrKh39exBE1LRU3svS5/bajC4y6G0MS8pYREeMDP5 C3EiRz0PasYqTCwxVhdQPaRVUv84gz7VePBXjGXCXv43gadma+jx1xwxcBapyNUbEfDF/7 SiDE5DKSbQQTPjTOvl2fMHqmqeE7a49z3ac3t0d7GrY0b1Rbrx/NVPewfWwyag== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1658499962; a=rsa-sha256; cv=none; b=j3eIhsMzUU/r1RNq0tAPIobpwZQzJ2LqCvtSJ6GUmQ80HsTlxSoFstmEwOvksADwgQhLVS XChuahHxCMLc5Esav+/2hgTExo2FbSQIHuSdGUiP/aLHlwjRxht9VFKRz7C3Bd+/uptmoS Ml9pYJ2UDMf7n/k+z9riqOFi/2yI7MKDTY1QIrm7nbhvcXxy90X+1SjPhUiArFXKUELuIp D/1bhHM1hmvUJDtz5jV2qy5pWWliR8f1i6HVSA8VssLR2ufqf4vuyINkXCA1BiAo113S24 ZPLU6k40hPL3h9Xg7i5Qe586PcX2VFzUzMtHpQ/X11VKqMcnpaz8aL4+VEt4dw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=fSgrX9zN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -6.74 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=fSgrX9zN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3FE50F2A1 X-Spam-Score: -6.74 X-Migadu-Scanner: scn0.migadu.com X-TUID: sP1qwbnOrLQD jgart writes: > Is Guix suitable for large monorepos? I use Guix against a large mono-repo, and there are a few difficulties. I should mention that I am not using Guix in the typical fashion with fixed commits (packages are pointed at HEAD and I don't use checksums). For my purposes, it's useful for me to build HEAD, although sometimes I use transformation options if I'm interested in a specific branch or commit. I've defined a package for the entire repo and then individual packages for the constituent projects which inherit from the repo package. For new commits: - Updating the source take a long time. - Indexing the entire repo takes a long time. - Copying over the entire tree to the build directory is unnecessary and takes time. Sometimes I want to build a local change I haven't committed anywhere, so I use the --with-source option. The problems are the same, but they're exacerbated. A happy medium I've landed on is to build some binaries outside of Guix and then use packages that are defined in terms of the binaries. These packages use patchelf to get everything linked correctly. This allows me to reify these packages into Guix while keeping the standard workflow from a programming language. I've also considered a phase that, after checkout, deletes all files not relevant to the project to lessen the burden of copying things over for building. So, in my opinion, these issues are not really Guix's fault, and it does a pretty good job all things considered. I think Guix used in a CI/CD pipeline (which I guess means cuirass) would handle mono-repositories wonderfully and do exactly what I want: reproducible, verifiable, builds. Guix as a development tool in these situations is still useful, but has some things that make it a little difficult to work with. Maybe someone has some ideas to improve things? -- Katherine