From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 5IX3EkG6HWEDLgEAgWs5BA (envelope-from ) for ; Thu, 19 Aug 2021 03:56:17 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id OEk+DkG6HWESCQAAB5/wlQ (envelope-from ) for ; Thu, 19 Aug 2021 01:56:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 487FEA7C2 for ; Thu, 19 Aug 2021 03:56:16 +0200 (CEST) Received: from localhost ([::1]:33254 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mGXHy-0008IE-95 for larch@yhetil.org; Wed, 18 Aug 2021 21:56:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45810) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGXHm-0008HO-DX for bug-guix@gnu.org; Wed, 18 Aug 2021 21:56:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46282) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mGXHm-0005WQ-6l for bug-guix@gnu.org; Wed, 18 Aug 2021 21:56:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mGXHm-0007an-19 for bug-guix@gnu.org; Wed, 18 Aug 2021 21:56:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#30290: guix-daemon slows to a crawl when a substitute server is offline Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 19 Aug 2021 01:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30290 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: zimoun Received: via spool by 30290-submit@debbugs.gnu.org id=B30290.162933810429112 (code B ref 30290); Thu, 19 Aug 2021 01:56:01 +0000 Received: (at 30290) by debbugs.gnu.org; 19 Aug 2021 01:55:04 +0000 Received: from localhost ([127.0.0.1]:57828 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGXGp-0007ZT-5J for submit@debbugs.gnu.org; Wed, 18 Aug 2021 21:55:04 -0400 Received: from mail-qt1-f179.google.com ([209.85.160.179]:44645) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGXGl-0007Yv-Bq for 30290@debbugs.gnu.org; Wed, 18 Aug 2021 21:55:01 -0400 Received: by mail-qt1-f179.google.com with SMTP id r21so3280473qtw.11 for <30290@debbugs.gnu.org>; Wed, 18 Aug 2021 18:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=yOU3dDgTsSk/jobRNeah8mqHIVuNQWSE7u9W/YoMiFI=; b=VDS/nnMlDk8M+zzcbFt8XTCbj3Pde5txNU11fgH+d1+cweCEjqpUA6RIv43fLUcYTe nrB8MEUdNa5yG42jXqoqn77SxFIo1pNYIDbrYphFZSi54DGl+65XTnoc6rDVfM1uiXvh Vh0pJtQAsKtL3q2svUqP6OvtrYkiysf0ZnN0SlC2rfkjSLF41oT6FFUcXXsDs30tGZyQ BiCgZIYc/PsvV4wpgsJmqLzCiYLz0eQ4HhQP86AWxKHKRD0FANRLV6HY2EYSXtA735Ye XdkE9MjNBIetRqB6orbEojOA5KB6lMnIbJtVXOTVDM8sWAP1Oi5uoAJvP49kWjBgcG+I WobQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=yOU3dDgTsSk/jobRNeah8mqHIVuNQWSE7u9W/YoMiFI=; b=ZX4AXgQYEArUgjQnmKMlxp11yLvy33uomdkc7HKp5PWtjvAQ3PIzRfqWj5UFFPGdDr W2lF1qWD7oPRhpHGrR4iYHqy47lt+gmtsJaWgpP/uJdiBOq3bXaSAiGjz5lHRPRkDsz8 UZc+l6mTVINVVjRuna/5DsyEOoUwm3Haea+u0qxIxS1qW88mzhesfodVuChBtBm8FhBj InCs/SnOq6+DzRxbxqUG2ohr1gK5usosnCB1y0jhHmXMH6Ce5Zrj4NEWwWPap7KXx7ql f9i/ELERAHIUAiCaKrtfuajd/ORrgIQKJpXHf3s4G5V3QjlpHuhsmo7NtC92TCpCETQH JK3A== X-Gm-Message-State: AOAM530rN9JDklt5tvTxIy/CAc8uMRfzNPlAGiBwa9Zsx47ItfilbAS8 VRnBStHO7XzbBWst7YTTx0ZlsXfeAp0= X-Google-Smtp-Source: ABdhPJzA60juAupW2ML45L76daZ1pdeK7WaUEzdWQTvI5LxOpK0//zWJhpvoe16ke3JNdyVvHQFTrA== X-Received: by 2002:a05:622a:c1:: with SMTP id p1mr10695535qtw.365.1629338093527; Wed, 18 Aug 2021 18:54:53 -0700 (PDT) Received: from hurd (dsl-236-123-187.b2b2c.ca. [207.236.123.187]) by smtp.gmail.com with ESMTPSA id c4sm697697qtd.62.2021.08.18.18.54.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Aug 2021 18:54:52 -0700 (PDT) From: Maxim Cournoyer References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> <87r1eqx569.fsf@gmail.com> Date: Wed, 18 Aug 2021 21:54:46 -0400 In-Reply-To: <87r1eqx569.fsf@gmail.com> (Maxim Cournoyer's message of "Wed, 18 Aug 2021 09:18:06 -0400") Message-ID: <87mtpew655.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mathieu Othacehe , 30290@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1629338176; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=yOU3dDgTsSk/jobRNeah8mqHIVuNQWSE7u9W/YoMiFI=; b=HNt7PIqJzBYQOc12MU26PyOxgVoqdsIDWBlibUv9TP8ogkOYO5PsgxBKUNCxwMtd5IQhI4 IBcXVx3BsFVTD9dUb7ruU035MNM2dcrHUfT6ylhN+Z+H8/rbio4/F3a8un7zkjOXRj1XRJ Kq4S08ZyzlTwiZvne086GSh1Jy7rvIRLIfS0KuifEkbkyI7IbXFn5hqWKDaBmyopjsIM09 KouPPVFFeT6y2O56wDBFfOKFnkxFLU2Nh1hy/z8ga70e3g6hszZxkigNvAmMrI+uQ/SaoI 5QqOpxwYpWjbCH2ASz/9qAaYYmNKpYIgyJLxdZoZfzo0550aaqCTconqiZSd9g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1629338176; a=rsa-sha256; cv=none; b=Ey0Ce2tuq5au//TcZMhpQQPZyIMP9YCz7IO4Qqf1H6965Hp12ilFO33uQypORR79Z75Aju Z6RlPWSBn398RldTmP7ZQNumNCpRfHJQiBljH7otXjE+sh+IYQN+t3dVALEEb2JMML8Lko 2W8adXZEaLr7hDxGQuAXPlehwVsmimcbkTON6X+T196O3hcGvUTJN8mPtQTCwdMHHvLQSS BIPiMEo8ke5nIBEx+YG+h1DaPUgRwjJvUhOaiQ8QW04t0H6+tB5KfeMx/lBDO0EFfswcXk i+455O51jRncKlOpNbckJQtDGE4jBctusjHjMLubI0nbZlIO0XpWgXBTSpdBeg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b="VDS/nnMl"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.32 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b="VDS/nnMl"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 487FEA7C2 X-Spam-Score: -1.32 X-Migadu-Scanner: scn0.migadu.com X-TUID: /ustV4NnOZIS Hello, Maxim Cournoyer writes: > Hi, > > And sorry for failing to produce a reply earlier :-). > > zimoun writes: > > [...] > >>>> From: Maxim Cournoyer >>>> Date: Fri, 18 Dec 2020 22:04:04 -0500 (24 weeks, 4 days, 18 hours ago) >>>> >>>> I'm not sure if the recent offloading work that Mathieu did touched th= at >>>> topic. I'd need to test the scenario. Perhaps a system test would be >>>> useful. >>>> ---------- >>>> >>>> From: Ludovic Court=C3=A8s >>>> Date: Tue, 22 Dec 2020 16:16:08 +0100 >>>> Date: Tue, 22 Dec 2020 16:16:08 +0100 (24 weeks, 1 day, 6 hours ago) >>>> >>>> Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 >>>> (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98= guix >>>> substitute --query=E2=80=99 process won=E2=80=99t retry connections to= an unreachable >>>> host. >>>> ---------- >>>> >>>> From: Efraim Flashner >>>> Date: Mon, 28 Dec 2020 14:19:02 +0200 >>>> Date: Mon, 28 Dec 2020 14:19:02 +0200 (23 weeks, 2 days, 9 hours ago) >>>> >>>> Occasionally my internet drops itself, and I find I'm left forever >>>> waiting for a timeout to see what sources I have cached locally. >>>> ---------- >>> >>> What is the current stats of this bug? Is it still happening with the >>> recent improvements of Cuirass? >> >> After reading all this, I think this bug can be closed. WDYT? > > Were you able to replay a scenario in which a substitute server is made > unreachable? That's the information that I'd like to have/see before > closing. I don't come across unreachable substitute servers often, and > can't think of a way to easily test this. > > I could make it hang by dropping the input/output connections with > iptables to a remote guix publish server, but then SSH also hangs, so > perhaps that's expected. > > I'll try to configure a couple local machines to act as publish servers, > and disconnect them from the network to see what happens. > > Thanks, > > Maxim I managed to get some problematic behavior from guix substitute: My test protocole was roughly like this: 1. Setup a 2nd machine (machine B) to act as a substitute server, and guix pull to the same commit as that of my main machine (machine A). 2. Run guix build -m manifest.scm on machine B (IP: 192.168.10.172). 3. On the machine A, run the command below, explicitly listing machine A as a substitute URL, along ci.guix.gnu.org. During a download from A, break the connection (I pulled the wifi USB dongle out): $ guix build -m ~/stow/guix/manifest.scm --substitute-urls=3D'http://192.16= 8.10.172 https://ci.guix.gnu.org' --no-offload substitute: updating substitutes from 'http://192.168.10.172:80'... 100.0% substitute: updating substitutes from 'http://192.168.10.172'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/lxm7brkbrkkv58c4kzlw1lh3wc0bm8wz-gimp-2.10.24.drv /gnu/store/ddv8jyzwk92nsg1dkv9n3scf6f7w83g5-keepassxc-2.6.6.drv /gnu/store/xky1y32mccplxsb448ziq68by2mvkdaz-ruby-asciidoctor-2.0.10.drv /gnu/store/0ph0sjib0d13n2fsl8w9prnky8g5fkvf-ruby-haml-5.0.4.drv /gnu/store/4dfwfj9qinw4vs6290gdy5qbnqbczm2v-ruby-temple-0.8.2.drv /gnu/store/b12krypa196yg6gzk2bvrh35i1fg5c7x-ruby-tilt-2.0.10.drv /gnu/store/131d8193hi1485ylnb9w8jm3jnlv3iyx-ruby-slim-4.1.0.drv /gnu/store/489nq0jqjby92kv3c6nwrdfqg45l40nw-ruby-sinatra-2.0.8.1.drv /gnu/store/yay3sa8nnq4j0ixwhp3bxfg5vfisfmf1-sflvault-client-0.9.2-1.8de3= 902.drv /gnu/store/2n1xyy0y3nnkrp3mpdifn8r7wf6pzpb0-sflvault-0.9.2-1.8de3902-che= ckout.drv /gnu/store/jsyhy4vxzr9yyg66kzk7w28xffyx050c-python-keyring-1.6.1.drv /gnu/store/kiwn3x2la23f1pa3a5ypsihhc6ja19y5-python-keyring-1.6.1-checkou= t.drv The following files will be downloaded: /gnu/store/2qphwngpvawl6f06d33b2jr18vk1hyc9-module-import-compiled /gnu/store/r7vsb0vl4y66jbq7b56zmrm60q2507zl-wireshark-3.4.7 /gnu/store/wnzx9anjdkmbnkcg5qdd3j77q1w2j1bd-yelp-3.32.2 /gnu/store/vcxwcwlwhvhxj15ma8ik8lghmz8sb2vq-vinagre-3.22.0 /gnu/store/yg8r6kz95p8v03gz0rglpwzrj21npzzw-spacefm-1.0.6 /gnu/store/bn35x60w72ad59a5pd7gmvxgjwgkqvag-youtube-dl-2021.06.06 /gnu/store/xkn540dzpz75hr9cx19xgd3b1r7vgswi-mpv-0.33.1 /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-desktop-4.2.5-debug /gnu/store/4h8ixlh5by2l09vv3rvknmlxv2gm9d6s-linphone-desktop-4.2.5 /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.2 /gnu/store/zczjaxs118155n3mx8w91c24izhx0h0f-ruby-asciimath-2.0.1 /gnu/store/zll4p79a29hw95d2gsh4vjdvd856ry4s-ruby-cucumber-html-formatter= -7.0.0 [...] /gnu/store/sa6hvh9bnw73mpplasbjb3idlv71rvcb-gnome-boxes-3.36.6 /gnu/store/6gy957mhm07zaa001avzkv2d8zhjdl5h-poppler-data-0.4.10 /gnu/store/7kwgmhlsy6qal56h3z19anxmw4c7pf35-diffoscope-177 /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1.18.2 /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.18.2 /gnu/store/f8hzmmnp8cm4yqq0y9cf7rgxl05hf423-cheese-3.38.0 substituting /gnu/store/7kwgmhlsy6qal56h3z19anxmw4c7pf35-diffoscope-177... substituting /gnu/store/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1... substituting /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.= 2... substituting /gnu/store/4h8ixlh5by2l09vv3rvknmlxv2gm9d6s-linphone-desktop-4= .2.5... downloading from http://192.168.10.172/nar/zstd/7kwgmhlsy6qal56h3z19anxmw4c= 7pf35-diffoscope-177 ... diffoscope-177 10.5M= iB/s 00:00 | 128KiB transferreddownloading from http://192.168.10.172/nar/z= std/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1 ... downloading from http://192.168.10.172/nar/zstd/f10an83xvya46ndh61y59qaw5vv= s5f7n-libreoffice-7.1.4.2 ... downloading from http://192.168.10.172/nar/zstd/4h8ixlh5by2l09vv3rvknmlxv2g= m9d6s-linphone-desktop-4.2.5 ... substitution of /gnu/store/4h8ixlh5by2l09vv3rvknmlxv2gm9d6s-linphone-deskto= p-4.2.5 complete substituting /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.= 18.2... downloading from http://192.168.10.172/nar/zstd/7bqpzvzanmvb4g1g6gqb1jmrw2j= 8gv3d-gst-plugins-bad-1.18.2 ... substitution of /gnu/store/7kwgmhlsy6qal56h3z19anxmw4c7pf35-diffoscope-177 = complete substituting /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1= .18.2... downloading from http://192.168.10.172/nar/zstd/hxvlcb4wgw0fpyi9ssc4x6f8w3h= lng55-gst-plugins-good-1.18.2 ... =20 ^ It hung up there, waiting indefinitely. What I would have expected instead, would have been to find out about the network failure, and retry from the other available substitute URL, else build locally. At that time, all the 'substitute' processes are blocked on a read(2) call, while one of the guix-daemon is also, and 2 others are blocked on select. That's not the same as the original report though; let's try to reproduce that one by running the same command again, while the substitute server B is still disconnected: --8<---------------cut here---------------start------------->8--- $ time guix build -m ~/stow/guix/manifest.scm --substitute-urls=3D'http://1= 92.168.10.172 https://ci.guix.gnu.org' --no-offload substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0%g= uix substitute: warning: 192.168.10.172: connection failed: No route to host substitute:=20 substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/lxm7brkbrkkv58c4kzlw1lh3wc0bm8wz-gimp-2.10.24.drv /gnu/store/ddv8jyzwk92nsg1dkv9n3scf6f7w83g5-keepassxc-2.6.6.drv /gnu/store/xky1y32mccplxsb448ziq68by2mvkdaz-ruby-asciidoctor-2.0.10.drv /gnu/store/0ph0sjib0d13n2fsl8w9prnky8g5fkvf-ruby-haml-5.0.4.drv /gnu/store/4dfwfj9qinw4vs6290gdy5qbnqbczm2v-ruby-temple-0.8.2.drv /gnu/store/b12krypa196yg6gzk2bvrh35i1fg5c7x-ruby-tilt-2.0.10.drv /gnu/store/131d8193hi1485ylnb9w8jm3jnlv3iyx-ruby-slim-4.1.0.drv /gnu/store/489nq0jqjby92kv3c6nwrdfqg45l40nw-ruby-sinatra-2.0.8.1.drv /gnu/store/yay3sa8nnq4j0ixwhp3bxfg5vfisfmf1-sflvault-client-0.9.2-1.8de3= 902.drv /gnu/store/2n1xyy0y3nnkrp3mpdifn8r7wf6pzpb0-sflvault-0.9.2-1.8de3902-che= ckout.drv /gnu/store/jsyhy4vxzr9yyg66kzk7w28xffyx050c-python-keyring-1.6.1.drv /gnu/store/kiwn3x2la23f1pa3a5ypsihhc6ja19y5-python-keyring-1.6.1-checkou= t.drv The following files will be downloaded: /gnu/store/2qphwngpvawl6f06d33b2jr18vk1hyc9-module-import-compiled /gnu/store/r7vsb0vl4y66jbq7b56zmrm60q2507zl-wireshark-3.4.7 /gnu/store/wnzx9anjdkmbnkcg5qdd3j77q1w2j1bd-yelp-3.32.2 /gnu/store/vcxwcwlwhvhxj15ma8ik8lghmz8sb2vq-vinagre-3.22.0 /gnu/store/yg8r6kz95p8v03gz0rglpwzrj21npzzw-spacefm-1.0.6 [...] /gnu/store/zvnnafb7hmiklj8wpvn9qdc85w8rdprl-gnucash-4.2-doc /gnu/store/rp2ai59zvx5m0k6db0cnkx6nn9n41qjd-gnucash-4.2 /gnu/store/hmy026sjdl489sy3i25r2kz9f70h3awm-gnucash-4.2-python /gnu/store/1bspzx0103mr17mxhgw0d9zdlgca2psq-spice-gtk-0.37 /gnu/store/bribnmf6djvh1d3rjr2vs5y97141ad97-osinfo-db-20201218 /gnu/store/r1a25sizf07nmh388ri4qybshzlcxbqd-libosinfo-1.7.1 /gnu/store/2z7p7ynamiarxkx4hnk8dk377xqgm3zl-tracker-2.3.5 /gnu/store/458bw9h0f0ybjdqwg4zm5gjjsmfxbalx-webkitgtk-2.32.3 /gnu/store/sa6hvh9bnw73mpplasbjb3idlv71rvcb-gnome-boxes-3.36.6 /gnu/store/6gy957mhm07zaa001avzkv2d8zhjdl5h-poppler-data-0.4.10 /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1.18.2 /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.18.2 /gnu/store/f8hzmmnp8cm4yqq0y9cf7rgxl05hf423-cheese-3.38.0 substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substituting /gnu/store/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1... substituting /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.= 2... substituting /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-desktop-4= .2.5-debug... substituting /gnu/store/bribnmf6djvh1d3rjr2vs5y97141ad97-osinfo-db-20201218= ... substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: warning: 192.168.10.172: connection failed: No route to ho= st downloading from https://ci.guix.gnu.org/nar/lzip/bribnmf6djvh1d3rjr2vs5y97= 141ad97-osinfo-db-20201218 ... osinfo-db-20201218 88KiB 5.9MiB/s 00= :00 [############# ] 73.1%substitution of /gnu/store/ns4n01xgbk6ccvd2z= 127v71d806rnr6f-inkscape-1.1 failed substitution of /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1= .4.2 failed substitution of /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-deskto= p-4.2.5-debug failed guix build: error: corrupt input while restoring archive from # real 1m13.549s user 0m25.348s sys 0m0.721s --8<---------------cut here---------------end--------------->8--- Hmm. Let's try again, --8<---------------cut here---------------start------------->8--- $ time guix build -m ~/stow/guix/manifest.scm --substitute-urls=3D'http://1= 92.168.10.172 https://ci.guix.gnu.org' --no-offload substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0%g= uix substitute: warning: 192.168.10.172: connection failed: No route to host substitute:=20 substitute: updating substitutes from 'http://192.168.10.172'... 0.0% The following derivations will be built: /gnu/store/lxm7brkbrkkv58c4kzlw1lh3wc0bm8wz-gimp-2.10.24.drv /gnu/store/ddv8jyzwk92nsg1dkv9n3scf6f7w83g5-keepassxc-2.6.6.drv /gnu/store/xky1y32mccplxsb448ziq68by2mvkdaz-ruby-asciidoctor-2.0.10.drv /gnu/store/0ph0sjib0d13n2fsl8w9prnky8g5fkvf-ruby-haml-5.0.4.drv /gnu/store/4dfwfj9qinw4vs6290gdy5qbnqbczm2v-ruby-temple-0.8.2.drv /gnu/store/b12krypa196yg6gzk2bvrh35i1fg5c7x-ruby-tilt-2.0.10.drv /gnu/store/131d8193hi1485ylnb9w8jm3jnlv3iyx-ruby-slim-4.1.0.drv /gnu/store/489nq0jqjby92kv3c6nwrdfqg45l40nw-ruby-sinatra-2.0.8.1.drv /gnu/store/yay3sa8nnq4j0ixwhp3bxfg5vfisfmf1-sflvault-client-0.9.2-1.8de3= 902.drv /gnu/store/2n1xyy0y3nnkrp3mpdifn8r7wf6pzpb0-sflvault-0.9.2-1.8de3902-che= ckout.drv /gnu/store/jsyhy4vxzr9yyg66kzk7w28xffyx050c-python-keyring-1.6.1.drv /gnu/store/kiwn3x2la23f1pa3a5ypsihhc6ja19y5-python-keyring-1.6.1-checkou= t.drv The following files will be downloaded: /gnu/store/2qphwngpvawl6f06d33b2jr18vk1hyc9-module-import-compiled /gnu/store/r7vsb0vl4y66jbq7b56zmrm60q2507zl-wireshark-3.4.7 /gnu/store/wnzx9anjdkmbnkcg5qdd3j77q1w2j1bd-yelp-3.32.2 [...] /gnu/store/sa6hvh9bnw73mpplasbjb3idlv71rvcb-gnome-boxes-3.36.6 /gnu/store/6gy957mhm07zaa001avzkv2d8zhjdl5h-poppler-data-0.4.10 /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1.18.2 /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.18.2 /gnu/store/f8hzmmnp8cm4yqq0y9cf7rgxl05hf423-cheese-3.38.0 substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% =20 substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substituting /gnu/store/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1... substituting /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.= 2... substituting /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-desktop-4= .2.5-debug... substituting /gnu/store/bribnmf6djvh1d3rjr2vs5y97141ad97-osinfo-db-20201218= ... substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: warning: 192.168.10.172: connection failed: No route to ho= st downloading from https://ci.guix.gnu.org/nar/lzip/bribnmf6djvh1d3rjr2vs5y97= 141ad97-osinfo-db-20201218 ... osinfo-db-20201218 88KiB 6.0MiB/s 00= :00 [############# ] 73.1%substitution of /gnu/store/ns4n01xgbk6ccvd2z= 127v71d806rnr6f-inkscape-1.1 failed substitution of /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1= .4.2 failed substitution of /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-deskto= p-4.2.5-debug failed guix build: error: corrupt input while restoring archive from # real 1m15.216s user 0m24.963s sys 0m0.702s --8<---------------cut here---------------end--------------->8--- Same thing, the daemon is still trying really hard to get something from that dead substitute server, slowing things down. That corrupted archive failure is curious, I wonder if it may be related. We'll have to keep this bug open I'm afraid :-/. Thanks, Maxim