From mboxrd@z Thu Jan 1 00:00:00 1970 From: ng0 Subject: Re: guix pull timing out on low resource servers Date: Tue, 16 May 2017 07:39:23 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:47256) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dAX52-00077o-95 for guix-devel@gnu.org; Tue, 16 May 2017 03:39:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dAX4x-000433-Ud for guix-devel@gnu.org; Tue, 16 May 2017 03:39:40 -0400 Received: from [195.159.176.226] (port=58014 helo=blaine.gmane.org) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dAX4x-00042T-Nq for guix-devel@gnu.org; Tue, 16 May 2017 03:39:35 -0400 Received: from list by blaine.gmane.org with local (Exim 4.84_2) (envelope-from ) id 1dAX4q-0000YV-6M for guix-devel@gnu.org; Tue, 16 May 2017 09:39:28 +0200 In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org On Mon, 15 May 2017, ng0 wrote: > On Mon, 15 May 2017, Andy Wingo wrote: > >> On Mon 15 May 2017 12:12, ng0 writes: >> >>> Since the switch to Guile 2.2 and guix pull taking much more >>> computing time and resources, all my virtual hosting servers >>> time out when I run guix pull on them without using mosh >>> and just openssh, because the process just takes much too >>> long. >>> >>> I doubt I'm the only one who experiences this, and just >>> increasing the server specs for this process is pointless >>> for what they run. >>> It's even worse than that, in the future there will be just >>> one DNS server running GuixSD, this requires minimal resources. >>> Should I just never update these devices or "just get a real >>> server"? >> >> I have the smallest DigitalOcean droplet size with a swap partition, an= d >> things work for me I think. Takes a while though. >> >> There will always be some machines that won't be able to "guix pull" >> because of resource constraints. But I think they are in a minority an= d >> they don't include e.g. the smallest DigitalOcean droplets. >> >> In the meantime we need to rework "build-all.scm" I think to not incur >> an O(N) memory usage in the size of the guix package set by >> topologically sorting the files (I know there are cycles, but the >> general approach should improve things), and by forking off and >> compiling the files in separate processes instead of doing everything >> from one process (albeit with many threads). Additionally there are >> some compiler speedups in Guile to be had (notably the "basic register >> allocation" task from >> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks). >> >> And in the meantime-meantime, the workaround is to use a swap file that >> is as large as necessary. >> >> Andy > > server 1: 1024MB RAM, 2048MB Swap, 4 cores. actually: 512MB RAM, 1024MB Swap. > server 2: 512MB RAM, 4096MB Swap, 2 cores. > > Didn't fail before.. of course I can try to increase the swap, but I doub= t=20 > that this really help. > > Both systems were able to pull before we used 2.2 > > I even compiled most of the system on one of them as substitutes weren't= =20 > available, so resources aren't the problem. > Great Scot=E2=80=A6 I recommend to create at least an 2.2GiB Swapfile when = you run=20 Guix on a virtualized machines (in case they don't have access to full=20 CPUs on their own. Be prepared for over 12 hours of 'guix pull' on those=20 low specs servers with the recent move to Guile 2.2, compared to just a couple of hours on bare metal.