unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* guix pull timing out on low resource servers
@ 2017-05-15 10:12 ng0
  2017-05-15 11:38 ` Andy Wingo
  2017-05-16 19:46 ` Leo Famulari
  0 siblings, 2 replies; 6+ messages in thread
From: ng0 @ 2017-05-15 10:12 UTC (permalink / raw)
  To: guix-devel

Since the switch to Guile 2.2 and guix pull taking much more
computing time and resources, all my virtual hosting servers
time out when I run guix pull on them without using mosh
and just openssh, because the process just takes much too
long.

I doubt I'm the only one who experiences this, and just
increasing the server specs for this process is pointless
for what they run.
It's even worse than that, in the future there will be just
one DNS server running GuixSD, this requires minimal resources.
Should I just never update these devices or "just get a real
server"?

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: guix pull timing out on low resource servers
  2017-05-15 10:12 guix pull timing out on low resource servers ng0
@ 2017-05-15 11:38 ` Andy Wingo
  2017-05-15 12:30   ` ng0
  2017-05-16 19:46 ` Leo Famulari
  1 sibling, 1 reply; 6+ messages in thread
From: Andy Wingo @ 2017-05-15 11:38 UTC (permalink / raw)
  To: ng0; +Cc: guix-devel

On Mon 15 May 2017 12:12, ng0 <ng0@pragmatique.xyz> writes:

> Since the switch to Guile 2.2 and guix pull taking much more
> computing time and resources, all my virtual hosting servers
> time out when I run guix pull on them without using mosh
> and just openssh, because the process just takes much too
> long.
>
> I doubt I'm the only one who experiences this, and just
> increasing the server specs for this process is pointless
> for what they run.
> It's even worse than that, in the future there will be just
> one DNS server running GuixSD, this requires minimal resources.
> Should I just never update these devices or "just get a real
> server"?

I have the smallest DigitalOcean droplet size with a swap partition, and
things work for me I think.  Takes a while though.

There will always be some machines that won't be able to "guix pull"
because of resource constraints.  But I think they are in a minority and
they don't include e.g. the smallest DigitalOcean droplets.

In the meantime we need to rework "build-all.scm" I think to not incur
an O(N) memory usage in the size of the guix package set by
topologically sorting the files (I know there are cycles, but the
general approach should improve things), and by forking off and
compiling the files in separate processes instead of doing everything
from one process (albeit with many threads).  Additionally there are
some compiler speedups in Guile to be had (notably the "basic register
allocation" task from
https://wingolog.org/archives/2016/02/04/guile-compiler-tasks).

And in the meantime-meantime, the workaround is to use a swap file that
is as large as necessary.

Andy

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: guix pull timing out on low resource servers
  2017-05-15 11:38 ` Andy Wingo
@ 2017-05-15 12:30   ` ng0
  2017-05-16  7:39     ` ng0
  0 siblings, 1 reply; 6+ messages in thread
From: ng0 @ 2017-05-15 12:30 UTC (permalink / raw)
  To: guix-devel

On Mon, 15 May 2017, Andy Wingo wrote:

> On Mon 15 May 2017 12:12, ng0 <ng0@pragmatique.xyz> writes:
>
>> Since the switch to Guile 2.2 and guix pull taking much more
>> computing time and resources, all my virtual hosting servers
>> time out when I run guix pull on them without using mosh
>> and just openssh, because the process just takes much too
>> long.
>>
>> I doubt I'm the only one who experiences this, and just
>> increasing the server specs for this process is pointless
>> for what they run.
>> It's even worse than that, in the future there will be just
>> one DNS server running GuixSD, this requires minimal resources.
>> Should I just never update these devices or "just get a real
>> server"?
>
> I have the smallest DigitalOcean droplet size with a swap partition, and
> things work for me I think.  Takes a while though.
>
> There will always be some machines that won't be able to "guix pull"
> because of resource constraints.  But I think they are in a minority and
> they don't include e.g. the smallest DigitalOcean droplets.
>
> In the meantime we need to rework "build-all.scm" I think to not incur
> an O(N) memory usage in the size of the guix package set by
> topologically sorting the files (I know there are cycles, but the
> general approach should improve things), and by forking off and
> compiling the files in separate processes instead of doing everything
> from one process (albeit with many threads).  Additionally there are
> some compiler speedups in Guile to be had (notably the "basic register
> allocation" task from
> https://wingolog.org/archives/2016/02/04/guile-compiler-tasks).
>
> And in the meantime-meantime, the workaround is to use a swap file that
> is as large as necessary.
>
> Andy

server 1: 1024MB RAM, 2048MB Swap, 4 cores.
server 2: 512MB RAM, 4096MB Swap, 2 cores.

Didn't fail before.. of course I can try to increase the swap, but I doubt 
that this really help.

Both systems were able to pull before we used 2.2

I even compiled most of the system on one of them as substitutes weren't 
available, so resources aren't the problem.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: guix pull timing out on low resource servers
  2017-05-15 12:30   ` ng0
@ 2017-05-16  7:39     ` ng0
  0 siblings, 0 replies; 6+ messages in thread
From: ng0 @ 2017-05-16  7:39 UTC (permalink / raw)
  To: guix-devel

On Mon, 15 May 2017, ng0 wrote:

> On Mon, 15 May 2017, Andy Wingo wrote:
>
>>  On Mon 15 May 2017 12:12, ng0 <ng0@pragmatique.xyz> writes:
>>
>>>  Since the switch to Guile 2.2 and guix pull taking much more
>>>  computing time and resources, all my virtual hosting servers
>>>  time out when I run guix pull on them without using mosh
>>>  and just openssh, because the process just takes much too
>>>  long.
>>>
>>>  I doubt I'm the only one who experiences this, and just
>>>  increasing the server specs for this process is pointless
>>>  for what they run.
>>>  It's even worse than that, in the future there will be just
>>>  one DNS server running GuixSD, this requires minimal resources.
>>>  Should I just never update these devices or "just get a real
>>>  server"?
>>
>>  I have the smallest DigitalOcean droplet size with a swap partition, and
>>  things work for me I think.  Takes a while though.
>>
>>  There will always be some machines that won't be able to "guix pull"
>>  because of resource constraints.  But I think they are in a minority and
>>  they don't include e.g. the smallest DigitalOcean droplets.
>>
>>  In the meantime we need to rework "build-all.scm" I think to not incur
>>  an O(N) memory usage in the size of the guix package set by
>>  topologically sorting the files (I know there are cycles, but the
>>  general approach should improve things), and by forking off and
>>  compiling the files in separate processes instead of doing everything
>>  from one process (albeit with many threads).  Additionally there are
>>  some compiler speedups in Guile to be had (notably the "basic register
>>  allocation" task from
>>  https://wingolog.org/archives/2016/02/04/guile-compiler-tasks).
>>
>>  And in the meantime-meantime, the workaround is to use a swap file that
>>  is as large as necessary.
>>
>>  Andy
>
> server 1: 1024MB RAM, 2048MB Swap, 4 cores.

actually: 512MB RAM, 1024MB Swap.

> server 2: 512MB RAM, 4096MB Swap, 2 cores.
>
> Didn't fail before.. of course I can try to increase the swap, but I doubt 
> that this really help.
>
> Both systems were able to pull before we used 2.2
>
> I even compiled most of the system on one of them as substitutes weren't 
> available, so resources aren't the problem.
>

Great Scot… I recommend to create at least an 2.2GiB Swapfile when you run 
Guix on a virtualized machines (in case they don't have access to full 
CPUs on their own. Be prepared for over 12 hours of 'guix pull' on those 
low specs servers with the recent move to Guile 2.2, compared to just a
couple of hours on bare metal.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: guix pull timing out on low resource servers
  2017-05-15 10:12 guix pull timing out on low resource servers ng0
  2017-05-15 11:38 ` Andy Wingo
@ 2017-05-16 19:46 ` Leo Famulari
  2017-05-17 12:39   ` Ludovic Courtès
  1 sibling, 1 reply; 6+ messages in thread
From: Leo Famulari @ 2017-05-16 19:46 UTC (permalink / raw)
  To: ng0; +Cc: guix-devel

On Mon, May 15, 2017 at 10:12:01AM +0000, ng0 wrote:
> Since the switch to Guile 2.2 and guix pull taking much more
> computing time and resources, all my virtual hosting servers
> time out when I run guix pull on them without using mosh
> and just openssh, because the process just takes much too
> long.

I have a super low-power server with 4 GB of RAM [0]. Building Guix with
Guile 2.2 takes about 100 minutes:


real    100m6.133s
user    0m18.804s
sys     0m2.724s

[0] This is basically an embedded router SOC, the AMD GX-412TC:
http://pcengines.ch/apu2b4.htm

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: guix pull timing out on low resource servers
  2017-05-16 19:46 ` Leo Famulari
@ 2017-05-17 12:39   ` Ludovic Courtès
  0 siblings, 0 replies; 6+ messages in thread
From: Ludovic Courtès @ 2017-05-17 12:39 UTC (permalink / raw)
  To: Leo Famulari; +Cc: guix-devel

Leo Famulari <leo@famulari.name> skribis:

> On Mon, May 15, 2017 at 10:12:01AM +0000, ng0 wrote:
>> Since the switch to Guile 2.2 and guix pull taking much more
>> computing time and resources, all my virtual hosting servers
>> time out when I run guix pull on them without using mosh
>> and just openssh, because the process just takes much too
>> long.
>
> I have a super low-power server with 4 GB of RAM [0]. Building Guix with
> Guile 2.2 takes about 100 minutes:
>
>
> real    100m6.133s
> user    0m18.804s
> sys     0m2.724s

One way or another, we should arrange so that users don’t have to pay
this cost.

Ludo’.

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2017-05-17 12:39 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-05-15 10:12 guix pull timing out on low resource servers ng0
2017-05-15 11:38 ` Andy Wingo
2017-05-15 12:30   ` ng0
2017-05-16  7:39     ` ng0
2017-05-16 19:46 ` Leo Famulari
2017-05-17 12:39   ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).