From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id +DdOBKo25mS00QAASxT56A (envelope-from ) for ; Wed, 23 Aug 2023 18:41:14 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id MDVGBKo25mR1LAAA9RJhRA (envelope-from ) for ; Wed, 23 Aug 2023 18:41:14 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 772F25B8CD for ; Wed, 23 Aug 2023 18:41:13 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="aoQOou2/"; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1692808873; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=5BrJEsOok7eA2nqlewEdLX/64ZR5CN72Ia189KxIJI8=; b=mAP7+Lym8wx111N4t4ty/6pegC34exjfo5q+wM+wxr7jbqdY+ScDBoKd9ZKeskEua5F+32 3i0exWNpoogWyX8xzyVm0ijqQUN+GKtlEwD5fcdE8gA9k1PxOR1I2hI9tX22iDJcGYKE8W km2e6DOpUxlK4ebyPG691ZKWE6S7Nn0Q51YGdCfAiAYh/XKhc3ka4aFoFp/YIPk/zfstjC 7PCY3lVJnY0Bq8dCX0fHQl99TFr2x2T0hGWUDYKsEdVE0H2Xb4vYjREThWi2Sxbvl0JH7n zEWoiBwgIOZd1Mg65faAD5tmWhl8qxX9vFyOlErDM2Xmi2a1H7bL0PNSnw1TJQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1692808873; a=rsa-sha256; cv=none; b=EFB7h8lAFsnGwYzMtkwLe8JO2pnqWcMX90PJs9mS7VDQTxbZRvi2od+s4ox9HUL+TSeHst GMb0Qjdchumki1E/12XLZaI1dYcbjT+UXhUs+260Ff0FrSyiH02ENVFz+Xi1Fb5I8TUXYw tlPkPZbyRqxuxa5XF6Vd/SHUB/3tzv8uubpSkgvy/yRW7ZUEjVemUawLZXV8ASlEMPw3kY 4lO6CYKfoJe8BKknSXFKONLu+Apsg3eZ5vMRSJijJxA2AK9LFQBlUcxM+NTeWv6AhCvwY/ iyYizTckoUXCjZbN6lAh3mTN1Vscy5JK/oYrEzwSyFIFj03pv6qGPq2xyu3BZg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="aoQOou2/"; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYquE-0002HM-Q1; Wed, 23 Aug 2023 12:40:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYqu7-0002G8-8I for guix-devel@gnu.org; Wed, 23 Aug 2023 12:40:23 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qYqu5-00045K-JM for guix-devel@gnu.org; Wed, 23 Aug 2023 12:40:23 -0400 Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-31979a6df33so1041391f8f.0 for ; Wed, 23 Aug 2023 09:40:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692808820; x=1693413620; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=5BrJEsOok7eA2nqlewEdLX/64ZR5CN72Ia189KxIJI8=; b=aoQOou2/k78sLUXL4++LqvarebXhe1W/KdDqezr/q+HQZXdAV3DdT4HdLrgVO0VmJI EZl7RYM9voZz0pxc3TrSh/0U8KDPSYpqNf74QM3oUHlt5oozUcLNFdXTdDRJdEoAHIn1 Llva1yKbSHL/JlI6RdOmELoZu0AsjP1nz0pcuYl0dRnOBE/mF8FiRB5hhO08BQxBWlOL Lt7AUm3hCAHjymroz3yJKZqBzQHx/ngv9ZrwIPSgWoST6apjXdanEIdswG85M1oQ5fFD gUsEYHXRG0c2aLjeWsm/ALCF5WuTDWJQJB1bd4f+R3nCLLl1yhXN0cS0K8PjIzGVoYfU SBNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692808820; x=1693413620; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5BrJEsOok7eA2nqlewEdLX/64ZR5CN72Ia189KxIJI8=; b=Tgw05b/ED6PZQUkpUqyRBIJe6fQn2xCi11efOZHtc5WTYKPlHUu+K9VzUlbB0i53sh Yhtn+wj7E0VUypaqdHz5PeEIUs8ZeGLc5FEJGV8v4Y70XoOUkWDX5iWGMYSVXB1oEpAg oRyHy47HrB1MIkO4N7RNfpyjvR83LqN3GROVeDx2yIW2LnMG+xV4FWP1NswP8xKqoYwB BAY7aKz/33SetELnEh9gGIXiFMIzatDJ/6g5MRUawcPveGQBu84d/gFzr16fmUuMLr01 uXC51bZU5hf8n+57wLLCWSTyfrKnbw0UfXfmbeBczAaVcZFHVlgg4hUGqphDVH7+woHU Wz+w== X-Gm-Message-State: AOJu0YzOr7Jp+aMTr2aMsLOrT+LDHuAQBcSumOqLjKleICRnCEX8TJy8 m2xk9IxJ5OWyi1upLW15u5xYd6Q4Szg= X-Google-Smtp-Source: AGHT+IF6Hpt91XhbqjBQusT2YPqOIaWBcsic6hFF2yNrIrK0ZDdDPRSroVB74BuMM8TMadmtHOCPlg== X-Received: by 2002:adf:e94a:0:b0:31a:d22c:1ee with SMTP id m10-20020adfe94a000000b0031ad22c01eemr8938549wrn.1.1692808819804; Wed, 23 Aug 2023 09:40:19 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id k9-20020adff289000000b003179d5aee67sm18071488wro.94.2023.08.23.09.40.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 09:40:19 -0700 (PDT) From: Simon Tournier To: Vagrant Cascadian , Nicolas Graves , Csepp Cc: guix-devel@gnu.org Subject: Slow guix pull In-Reply-To: <875y7c3jnc.fsf@wireframe> References: <87h6qzzll7.fsf@ngraves.fr> <87h6qxyv2m.fsf@riseup.net> <87wmztqd8t.fsf@ngraves.fr> <875y7c3jnc.fsf@wireframe> Date: Wed, 23 Aug 2023 15:48:06 +0200 Message-ID: <86o7ix3m5l.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -9.58 X-Spam-Score: -9.58 X-Migadu-Queue-Id: 772F25B8CD X-Migadu-Scanner: mx1.migadu.com X-TUID: MrfgvVFT6q0i Hi, On Sat, 24 Jun 2023 at 09:33, Vagrant Cascadian wrote: > So, presuming substitutes are available, the main slowness with guix > seems to be guix pull? I agree (and for the record, I would add in addition =E2=80=9Cguix search= =E2=80=9D too, another story :-)). Well, from my understanding about =E2=80=9Cguix pull=E2=80=9D, the first bo= ttleneck is about =E2=80=9CComputing Guix derivation for ...=E2=80=9D from build-aux/bu= ild-self.scm. Maybe I misread something and for sure I am probably missing some details. On my desktop machine, that part takes 84 seconds, roughly. >From my understanding, it=E2=80=99s because of this procedure, --8<---------------cut here---------------start------------->8--- (define (proxy input output) "Dump the contents of INPUT to OUTPUT until EOF is reached on INPUT. Display a spinner when nothing happens." (define spin (circular-list "-" "\\" "|" "/" "-" "\\" "|" "/")) (setvbuf input 'block 16384) (let loop ((spin spin)) (match (select (list input) '() '() 1) ((() () ()) (when (isatty? (current-error-port)) (display (string-append "\b" (car spin)) (current-error-port)) (force-output (current-error-port))) (loop (cdr spin))) (((_) () ()) ;; Read from INPUT as much as can be read without blocking. (let ((bv (get-bytevector-some input))) (unless (eof-object? bv) (put-bytevector output bv) (loop spin))))))) --8<---------------cut here---------------end--------------->8--- that is called by, ;; Wait for a connection on SOCK and proxy build output so it can be ;; processed according to the settings currently in effect (build ;; traces, verbosity level, and so on). (match (accept sock) ((port . _) (close-port sock) (delete-file node) (proxy port (current-build-output-port)))) And that dump is currently on strong annoyance, IMHO. Because most of the other derivations can be substituted but this part cannot, again from my poor understanding. Note that this part can be very painful. For example, on my laptop from 2016, it takes ~180 seconds. Well, 3 minutes for =E2=80=9Cdumping content= =E2=80=9D=E2=80=A6 that=E2=80=99s a piece of content! :-) Well, I do not know exactly which kind of content is living at =E2=80=99por= t=E2=80=99 and then if =E2=80=99(current-build-output-port)=E2=80=99 needs all this co= ntent. Therefore, I do not know if something could be trimmed=E2=80=A6 but I guess something could be improved here. Cheers, simon