From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id CLDcM7ADe2BmwwAAgWs5BA (envelope-from ) for ; Sat, 17 Apr 2021 17:50:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id uBx7LbADe2AYHgAAbx9fmQ (envelope-from ) for ; Sat, 17 Apr 2021 15:50:08 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 70C17B85A for ; Sat, 17 Apr 2021 17:50:08 +0200 (CEST) Received: from localhost ([::1]:47598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXnCx-00005L-Hc for larch@yhetil.org; Sat, 17 Apr 2021 11:50:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49942) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXnCm-000059-K4 for guix-devel@gnu.org; Sat, 17 Apr 2021 11:49:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:35400) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXnCm-0007aJ-DH; Sat, 17 Apr 2021 11:49:56 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41656 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lXnCk-0005KM-St; Sat, 17 Apr 2021 11:49:55 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mekeor Melire Subject: Re: Speed up package installation by using images instead of archives (like distri)? References: X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 28 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 17 Apr 2021 17:49:52 +0200 In-Reply-To: (Mekeor Melire's message of "Sun, 04 Apr 2021 13:59:18 +0200") Message-ID: <878s5gud0f.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1618674608; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=2C/Kq+LMVo15mWQb5Aipst9yQh32txiJOqciZE/tYIU=; b=Rl0ilbmYZO3fx244kRltvpPW5wQfMbsFUo1iV+CNbkcKIb7mqWjT9Hba3oxp3wVCRRfcqo xwjL4/6VvxvdB7/4nC1oRYNRby9IlkXp1+XEgizaqF1nT+NriP3EVgfi7nXpYsdwNrMDY/ djNsmStKKVAvRoTsHxbinEVua/iBXgBqvpsOJVXSoG7eoz5rgVv4kKkX45cEXMSbinsN+Y xjHuY4f6rp2KW5IRC4R1aR7n/1a4W9dDgtPRUm3PF8TkMEgw5FnJoAXb2AUojCB0Q1QKR6 d3MGBpMvJspY370+3fpBGQGI1Obrm7mH6JHCdptawDSBNIt2t8jG8njTL+un+Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1618674608; a=rsa-sha256; cv=none; b=LbGozsCncQGEmMO10ziLBU76IJaS4kQEjbnVFlNlHE5SwGeSP/fjUaO5Pq4a8w9GynU49o AKk8xebviafO4OpY+nqZ2SDDc/KQqOhZyp/5crLNTyXGEQ9NhLSnYYpo1xZemiTCGKhk4H ++awB0dlboJVerHXtXXRWWI9uC3T3dghWNejsQcpizSM9oyN4c1KBP+7qoTna9kyzV4R+m NNF96j3Z5FxR3hdMXxIUWGeyjy/gAJABzK1KK+smp4OL10NSGQaSetTlJME3uKQGGDs+6S d6YZimn5RXZPc/U6lvlEu2IhCP7dQ/8MXi4rvGa908uCf6rTWaC000EudGYaLg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -0.44 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 70C17B85A X-Spam-Score: -0.44 X-Migadu-Scanner: scn0.migadu.com X-TUID: T+X7fKabvYSC Hi! Mekeor Melire skribis: > In contrast, distri downloads the package as an (squashfs) > image. That's it. The image will then be mounted (which is not an > expensive IO operation) as a read-only virtual-file-system. (But as > it's not recommendable to mount as many images as there are installed > packages, distri mounts the package-images lazily, i.e. right before > they are being used.) > > (At least, the above explanation reflects my understanding.) > > Thus, I wonder: (0) Does it make sense to make Guix follow this idea > as well? I.e. should we make Guix install packages by downloading > images and mounting those instead of extracting archives? (1) Does > this even fit into Guix technically? I.e. how hard would it be to > implement this? (2) And would it even be worth it? I.e. how much > faster would Guix become? The Guix blog post mentions squashfs images, suggesting it=E2=80=99s not transposable to Guix. :-) Specifically, there are several practical issues if you think about what it would take to mount squashfs images instead of extracting things: you=E2=80=99d have one mount point per store item? how does that interact w= ith GC? what about GNU/Hurd support? I view it as a can of worms, if it=E2=80= =99s workable at all. Would it be faster? Like other proposals in the distri talk, it=E2=80=99s = all about doing things lazily instead of eagerly. In this case, decompression happens lazily instead of eagerly. I think that could make application startup on a cold cache slower, precisely because that work has to happen anyway. More importantly, mounting all these squashfs images at boot time could be costly. Looking at the big picture, with zstd substitutes available, the main speed issues in Guix are elsewhere. What would make a big difference is if we didn=E2=80=99t have to download (or build) that much so often. This = is what the end of the blog post hints at. HTH! Ludo=E2=80=99.