From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id yCXhCBEfu2EBdAAAgWs5BA (envelope-from ) for ; Thu, 16 Dec 2021 12:12:17 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id eCWaBBEfu2EKJgAAB5/wlQ (envelope-from ) for ; Thu, 16 Dec 2021 11:12:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A1F5930D1 for ; Thu, 16 Dec 2021 12:12:16 +0100 (CET) Received: from localhost ([::1]:37246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxogJ-0008Mk-Q9 for larch@yhetil.org; Thu, 16 Dec 2021 06:12:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:57080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxofx-0008KW-HT for guix-devel@gnu.org; Thu, 16 Dec 2021 06:11:53 -0500 Received: from [2a00:1450:4864:20::429] (port=37713 helo=mail-wr1-x429.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mxofv-0007o1-Ns for guix-devel@gnu.org; Thu, 16 Dec 2021 06:11:53 -0500 Received: by mail-wr1-x429.google.com with SMTP id t26so7182887wrb.4 for ; Thu, 16 Dec 2021 03:11:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=xV+hHJ7XQ3tmjLVf8oKmpJ1jUXrtMkg4XdsryOXfSqI=; b=n6N7CQ2qFJ20CivEPRNiGKheseYMPuz62m3wepd+p1hLbYPtkr/Y+BL6mV34agphTw UcW/1llaeJ8OMWj2au5QujNxzSOpmTTPTouuuueFcJROpY/kpVe0NWSwe5re17popi1d c8L6D9k5yMbKGuoruHgn0JfVxfxbqAFoevAjOEWJzPvRsOEAhFOu+LjOyiOg32ThLC5Z CK9SjnIdB3c2VFKO4BORpN9EzBIp/IlOyiB3AS7Vpkazn+AnLR0ZfheyzelnudfI2exa BCfeGVG90zm0kxzSaVzMTH7Mz52ScrJnhgRpMWfhJoE5xoa/2824IV2UdPlYKjJcvnW6 s4WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=xV+hHJ7XQ3tmjLVf8oKmpJ1jUXrtMkg4XdsryOXfSqI=; b=NK4rR7Q59mcBCRjy/R4a8yoHt37LPWu/fxc+sJlGgGKsukyXGuwzu+5Zy4nNyVmqY2 V5miDXqPDXamkgexwD9uVQAOYAk7XaaTOfbgehWoQha8VKtEklgCQogS4i7LRIfxoVn2 R8ldemakHAf1AM9ckpPNyrMFDkaqE/7Py2ITxW885MTn9/JLiavKuMGnrFXVgvHrKTFU tK6iv3O7655tJC2RGf8m5Dm8GJWjgyyhQ2xu+ayVYTVS1pF9tO0Sp4av1jcOam+vrKyT Cj3ZTYttjbV9HHF0Wy6UGZ630IOxhDc7DtjlkdFSv5xX73/70pL1wn6DETi3osne5wwK D9Aw== X-Gm-Message-State: AOAM530Vu2TH/Z0RDgRrvC5wJ7Mz/JhKIMQ6ZGbVUAOg2K3pLglQ2mYa 4lYdhNykxb20O7+ydF+S8aVQlwCDINs= X-Google-Smtp-Source: ABdhPJzA1Dc9UZ4t50flXOjf2l1dohnUF5xGU+E4NtWctUw4+YOxL+6o1YEzCUTp69AuZI9dYgleXg== X-Received: by 2002:a5d:6289:: with SMTP id k9mr1445631wru.501.1639653109921; Thu, 16 Dec 2021 03:11:49 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id p5sm4672224wrd.13.2021.12.16.03.11.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Dec 2021 03:11:49 -0800 (PST) From: zimoun To: Christopher Baines Subject: Re: Mid-December update on bordeaux.guix.gnu.org In-Reply-To: <87sfutxstt.fsf@cbaines.net> References: <874k79zs29.fsf@cbaines.net> <86r1adiiv4.fsf@gmail.com> <87sfutxstt.fsf@cbaines.net> Date: Thu, 16 Dec 2021 12:05:11 +0100 Message-ID: <86k0g4izdk.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::429 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::429; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x429.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639653136; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=xV+hHJ7XQ3tmjLVf8oKmpJ1jUXrtMkg4XdsryOXfSqI=; b=NvJe+TlNeAhns8I7hk7GbXK22DF6uwVf55KUIVFA0BG/9gEQ84bvyobejc0saQFzlqgPKT ayd7yuBUePwc6AbIzHFxFCLvxCD84/ZXIG+X2Jqrd370Lb0HiZcIvtKjlbXjW2EHhLwmhu xqlA8otDoGYh6iBzCMevRjk/zIZ0UUdaGxIq2/HkGZDg1Nht0E5aCJgBv1libg2iWlYWG1 NuiU8rlT6D5idDwuNAH2fz65PeRUbTy67gBFmKkTpTSKgqitj2utYV9sJ7SwJxsW0qle1Z nGNKQjyy45LHBeL/ojvGhUW12rFD+sSZT+BbGyNo+nq554BXSNHTgKmyi2FN4g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1639653136; a=rsa-sha256; cv=none; b=qnscEDCOXOWFNnrkCXgsaSqa7TSnfNiRM0PCbmhVdqTl3SMr1PYB6fIKxLQOai7SD1v6v1 hpLVaDhv/C7FBgUbX40Tt4nHLIQ4gDyOU6EVUEtGKHJABnAmWej9j4oNht+RQd4HfDJX7s m4HbmoCzfb6Je91fTz7/2M7hf+2SO2/81W86c7JwGNq2nw/yfQppzFfT5WwPIiPliQBxwo zJ+x3q2zZ/C6HoaFqRE+qTpjdHGnVfGoZsqnrpp1VovDCtG4uIsIbM2GqHOS2MjgOMY7Ig MK/l/3QiYMcsZ7WZgHdT/zjxyelrtnUWwJ6uQuNPKEoTAx1n0aF3LIwJP4sqEg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=n6N7CQ2q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.19 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=n6N7CQ2q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: A1F5930D1 X-Spam-Score: -4.19 X-Migadu-Scanner: scn0.migadu.com X-TUID: 6zd/cXYsdip5 Hi Chris, On Thu, 16 Dec 2021 at 00:20, Christopher Baines wrote: > zimoun writes: >> Do you think that Bordeaux could run >> >> > > The Guix Build Coordinator just builds derivations. I haven't got it to > build a manifest before, but that's possible I guess. I am not sure to understand since Cuirass also builds derivations and the purpose of this source-manifest.scm is to let Cuirass ingest all sources, > I think it's unnecessary though, since I believe derivations for all > origins of all packages are already being built, that happens through > just asking the coordinator to build derivations for all packages, you > don't need to specify "source" derivations separately. Your assumption is wrong, IMHO. We have many failed examples and at the end Ludo wrote this source-manifest.scm to be sure that all is ingested for sure. >> ? Having a redundancy about all origins would avoid breakage. For >> instance, because Berlin was down yesterday morning, =E2=80=9Cguix pull= =E2=80=9D was >> broken because the missing =E2=80=99datefuge=E2=80=99 package =E2=80=93 = disappeared upstream. > > I would hope that bordeaux.guix.gnu.org has a substitute for that, could > you check the derivation against data.guix.gnu.org, and see if there's a > build? Use a URL like: > > https://data.guix.gnu.org/gnu/store/vhj3gg00hzqfi8lazr3snb9msr4a3q6l-da= tefudge_1.23.tar.xz.drv > > There is one issue though, bordeaux.guix.gnu.org doesn't provide content > addressed files in the same way guix publish does. I hope to add that > through the nar-herder, and once that's added, bordeaux.guix.gnu.org can > hopefully be added to the list of content addressed mirrors: > > https://git.savannah.gnu.org/cgit/guix.git/tree/guix/download.scm#n368 > > That would mean that the bytes for a tar archive for example would be > available by the sha256 hash, not just as a nar. I'm not sure to what > extent this would help, but it's probably useful. Thanks for explaining the details. From a pragmatical point of view as end-user, =E2=80=9Cguix pull=E2=80=9D must Just Work whatever the plumbing. For instance, some of us spent energy to =E2=80=9Cevangelize=E2=80=9D in sc= ientific communities how Guix is awesome. Next morning, people give a look and bang! Because a tiny and short outage. Obviously, =E2=80=9Cshit happens= =E2=80=9D(*) and it is really really really sparse but too late the damage is there. My feeling here is that both build farms work independently instead of coordinate the usage of resources and exploit this strength to have two. (*) quoting Forest Gump: :-) =E2=80=93 whoa man, you just ran through a big pile of dog shit! =E2=80=93 it happens =E2=80=93 what? shit? =E2=80=93 sometimes >> I remember discussions about CDN [2,3,4,5,6]. I do not know if it >> solves the issue but from my understanding, it will improve at least >> performance delivery. Well, it appears to me worth to give a try. >> >> 2: >> 3: >> 4: >> 5: >> 6: > > Effectively this is moving towards building a CDN. With the nar-herder, > you could deploy reverse proxies (or edge nodes) in various > locations. Then the issue just becomes how to have users use the ones > that are best for them. This might require doing some fancy stuff with > GeoIP based DNS, and somehow sharing TLS certificates between the > machines, but I think it's quite feasible. Considering the human resource vs the money resource, it appears to me better to invest the human energy into things that do not exist and rely, for now, on existing solutions; even if the project has to pay money. For what my opinion is worth here. >> To me, one first general question about backup coordination is to define >> a window for time: >> >> - source: forever until the complete fallback to SWH is robust; >> - all the substitutes to run =E2=80=9Cguix time-machine --commit=3D<> -= - help =E2=80=9D >> for any commit reachable by inferior: forever; >> - package substitute: rule something. > > The idea I've been working with so far is simply to store everything > that's built, forever. For sure, anyone wants that at the end. My point is raising the priority of the intermediary steps. > Currently, that amounts to 561,043 nars totaling ~2.5TB's. > > How feasible this is depends on a number of factors, but I don't have > any reason to think it's not feasible yet. That=E2=80=99s exactly my point! It is not about feasibility =E2=80=93 all= is doable with enough time and energy =E2=80=93 but instead it is about controlling t= he factors to =E2=80=9Crobustify=E2=80=9C what the project consider highly imp= ortant =E2=80=93 as keep all sources or never break =E2=80=98guix pull=E2=80=99 =E2=80=93 whate= ver the status of infrastructure. Again, thanks for all the work. Becausse, in any case, for sure, the situation is daily improving. :-) Cheers, simon