all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Guix pull: avoiding "Computing Guix derivation"
@ 2024-05-13 15:04 Edouard Klein
  2024-05-13 15:45 ` Ryan Sundberg
                   ` (2 more replies)
  0 siblings, 3 replies; 16+ messages in thread
From: Edouard Klein @ 2024-05-13 15:04 UTC (permalink / raw)
  To: guix-devel

Hi Guix,

First, I'd like to apologize for not having taken the time to answer
those who helped me on a previous guix performance issue (with
containers), the reason is tied to the topic of this email: the store
has eaten all the space of my server, and solving that takes precedence
over everything else, because no space == no services.

So, I need to clear some space, and to do that I need to have every user
run guix pull (and by that I mean root will sudo -u $user guix pull),
then update all of their profiles, and then guix gc
--delete-generations.

This ought to turn deduplication up to 11, and enjoy a reduced store
size.

I've already solved the cache size problem:
mount -t overlay overlay -o \
lowerdir="/root/.cache/guix",upperdir="/home/$user/.cache/guix-overlay",workdir="/home/$user/.cache/guix-workdir" \
"/home/$user/.cache/guix"

Then
bindfs --mirror=$user /home/$user/.cache/guix /home/$user/.cache/guix

This lets root (who just ran guix pull) share its cache with every user,
and avoid blasting away 700MB of disk space in every $HOME to reproduce
the cache.

However, now, I'm facing the previously addressed problem of guix pull
being slow and hungry:
https://www.mail-archive.com/guix-devel@gnu.org/msg66442.html (Guix pull speed)
https://yhetil.org/guix/87h6mwf4u3.fsf@lapenas.dev/T/ (guix pull
performance)

On my server, in order to run guix pull, I have to stop all other
services otherwise I run out of ram.

Then, one root has pulled, I need to wait 4 minutes /per user/ for guix
pull to finish its "Computing Guix derivation" step.

I would like to know two things, one is for the sake of knowledge, and
the other is to solve the problem at hand:

- Why is this step not substitutable ? The inputs are known, a hash can
be derived, a substitute server could be queried for an output of that
hash ? What am I missing ? Does the guix derivation not end up in the
store ? What makes it so special that it can't be served by a substitute
server ?

- Is there a way (even a very dirty one, like hand copying stuff accross
  /var/guix/profiles/per-user/*/current-guix) I can stop paying this 4
  minutes per user price ? As I said, this is downtime on my server, as
  I need to stop all other services to let guix pull finish.

Thanks in advance.

Sorry for beating a dead horse, it's just that I can't scale anything up
until I solve these performance issues. Sure I could rent a bigger
server, but that's just kicking the can down the road.


Cheers,

Edouard.


^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2024-06-24 15:28 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-13 15:04 Guix pull: avoiding "Computing Guix derivation" Edouard Klein
2024-05-13 15:45 ` Ryan Sundberg
2024-05-14 14:32   ` Edouard Klein
2024-06-24 15:25   ` Edouard Klein
2024-05-13 19:28 ` Simon Tournier
2024-05-13 21:11   ` Richard Sent
2024-05-13 23:30     ` Simon Tournier
2024-05-14  0:52       ` Richard Sent
2024-05-14  9:47         ` Simon Tournier
2024-05-14 14:40           ` Edouard Klein
2024-05-14 10:12 ` Ludovic Courtès
2024-05-14 14:02   ` Richard Sent
2024-05-14 17:29     ` Richard Sent
2024-05-14 14:48   ` Edouard Klein
2024-05-15  8:38     ` Josselin Poiret
2024-05-31 14:02       ` Edouard Klein

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.