* Speed up package installation by using images instead of archives (like distri)?
@ 2021-04-04 11:59 Mekeor Melire
2021-04-17 15:49 ` Ludovic Courtès
0 siblings, 1 reply; 2+ messages in thread
From: Mekeor Melire @ 2021-04-04 11:59 UTC (permalink / raw)
To: guix-devel
Hello Guix!
On the Guix-blog¹, Ludo already mentioned the lightning talk of Michael
Stapelberg on the “distri” package manager which installs packages
faster than common package managers by an order of magnitude.
Common package managers install a package by downloading it as an
(compressed) archive and extracting the archive. But this extraction
takes very long as it requires to create dozens of files on the file
system which is an expensive IO operation.
In contrast, distri downloads the package as an (squashfs) image. That's
it. The image will then be mounted (which is not an expensive IO
operation) as a read-only virtual-file-system. (But as it's not
recommendable to mount as many images as there are installed packages,
distri mounts the package-images lazily, i.e. right before they are
being used.)
(At least, the above explanation reflects my understanding.)
Thus, I wonder: (0) Does it make sense to make Guix follow this idea as
well? I.e. should we make Guix install packages by downloading images
and mounting those instead of extracting archives? (1) Does this even
fit into Guix technically? I.e. how hard would it be to implement this?
(2) And would it even be worth it? I.e. how much faster would Guix
become?
I hope for an exciting discussion!
mekeor
--
Footnotes:
1:
https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Speed up package installation by using images instead of archives (like distri)?
2021-04-04 11:59 Speed up package installation by using images instead of archives (like distri)? Mekeor Melire
@ 2021-04-17 15:49 ` Ludovic Courtès
0 siblings, 0 replies; 2+ messages in thread
From: Ludovic Courtès @ 2021-04-17 15:49 UTC (permalink / raw)
To: Mekeor Melire; +Cc: guix-devel
Hi!
Mekeor Melire <mekeor@posteo.de> skribis:
> In contrast, distri downloads the package as an (squashfs)
> image. That's it. The image will then be mounted (which is not an
> expensive IO operation) as a read-only virtual-file-system. (But as
> it's not recommendable to mount as many images as there are installed
> packages, distri mounts the package-images lazily, i.e. right before
> they are being used.)
>
> (At least, the above explanation reflects my understanding.)
>
> Thus, I wonder: (0) Does it make sense to make Guix follow this idea
> as well? I.e. should we make Guix install packages by downloading
> images and mounting those instead of extracting archives? (1) Does
> this even fit into Guix technically? I.e. how hard would it be to
> implement this? (2) And would it even be worth it? I.e. how much
> faster would Guix become?
The Guix blog post mentions squashfs images, suggesting it’s not
transposable to Guix. :-)
Specifically, there are several practical issues if you think about what
it would take to mount squashfs images instead of extracting things:
you’d have one mount point per store item? how does that interact with
GC? what about GNU/Hurd support? I view it as a can of worms, if it’s
workable at all.
Would it be faster? Like other proposals in the distri talk, it’s all
about doing things lazily instead of eagerly. In this case,
decompression happens lazily instead of eagerly. I think that could
make application startup on a cold cache slower, precisely because that
work has to happen anyway. More importantly, mounting all these
squashfs images at boot time could be costly.
Looking at the big picture, with zstd substitutes available, the main
speed issues in Guix are elsewhere. What would make a big difference is
if we didn’t have to download (or build) that much so often. This is
what the end of the blog post hints at.
HTH!
Ludo’.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-04-17 15:50 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-04 11:59 Speed up package installation by using images instead of archives (like distri)? Mekeor Melire
2021-04-17 15:49 ` Ludovic Courtès
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.