* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull
@ 2020-02-27 20:06 Danny Milosavljevic
2022-07-13 14:00 ` Maxim Cournoyer
0 siblings, 1 reply; 6+ messages in thread
From: Danny Milosavljevic @ 2020-02-27 20:06 UTC (permalink / raw)
To: 39813
[-- Attachment #1: Type: text/plain, Size: 244 bytes --]
Hi,
I just got a report that with Guix in a virtual Machine (like described in the
manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM.
It does work fine with 4 GB of RAM.
[1] build .....guix-packages-base.drv failed
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull
2020-02-27 20:06 bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull Danny Milosavljevic
@ 2022-07-13 14:00 ` Maxim Cournoyer
2022-07-13 16:58 ` Csepp
0 siblings, 1 reply; 6+ messages in thread
From: Maxim Cournoyer @ 2022-07-13 14:00 UTC (permalink / raw)
To: Danny Milosavljevic; +Cc: 39813-done
Hi Danny,
Danny Milosavljevic <dannym@scratchpost.org> writes:
> Hi,
>
> I just got a report that with Guix in a virtual Machine (like described in the
> manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM.
> It does work fine with 4 GB of RAM.
I don't see any reference of 1 GiB being enough in our current version
of the manual. If you do, please let me know.
Closing for now.
Thanks,
Maxim
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull
2022-07-13 14:00 ` Maxim Cournoyer
@ 2022-07-13 16:58 ` Csepp
2022-07-13 17:21 ` Julien Lepiller
0 siblings, 1 reply; 6+ messages in thread
From: Csepp @ 2022-07-13 16:58 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: dannym, 39813, 39813-done
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
> Hi Danny,
>
> Danny Milosavljevic <dannym@scratchpost.org> writes:
>
>> Hi,
>>
>> I just got a report that with Guix in a virtual Machine (like described in the
>> manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM.
>> It does work fine with 4 GB of RAM.
>
> I don't see any reference of 1 GiB being enough in our current version
> of the manual. If you do, please let me know.
>
> Closing for now.
>
> Thanks,
>
> Maxim
I think it's enough if you only use a single core.
If any guix operations goes out of memory, add --cores=1.
So: guix pull --cores=1
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull
2022-07-13 16:58 ` Csepp
@ 2022-07-13 17:21 ` Julien Lepiller
2022-07-13 20:00 ` Liliana Marie Prikler
0 siblings, 1 reply; 6+ messages in thread
From: Julien Lepiller @ 2022-07-13 17:21 UTC (permalink / raw)
To: 39813, raingloom, maxim.cournoyer; +Cc: dannym, 39813-done
[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]
I've heard that theory before. From observation on my late armhf server (two cores):
- it takes just below 2GB to build one of the derivations
- It doesn't swap a single byte
- whether with two or a single core, it takes roughly the same amount of memory
- substitution is nice, it doesn't require lots of memory (and then --cores is useless)
I think it's because we load all the files in a batch before we build them. The biggest amount of memory required is not for running the compiler on a thread, but for loading files and keeping them in memory for the whole duration of the build. With more threads, we still don't load each file more than once (twice to build it), so there's no reason it should take more memory.
Or maybe the process of loading and building is inherently single-threaded? I don't think so, but maybe?
Or it doesn't honor --cores.
Le 13 juillet 2022 18:58:58 GMT+02:00, Csepp <raingloom@riseup.net> a écrit :
>
>Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> Hi Danny,
>>
>> Danny Milosavljevic <dannym@scratchpost.org> writes:
>>
>>> Hi,
>>>
>>> I just got a report that with Guix in a virtual Machine (like described in the
>>> manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM.
>>> It does work fine with 4 GB of RAM.
>>
>> I don't see any reference of 1 GiB being enough in our current version
>> of the manual. If you do, please let me know.
>>
>> Closing for now.
>>
>> Thanks,
>>
>> Maxim
>
>I think it's enough if you only use a single core.
>If any guix operations goes out of memory, add --cores=1.
>So: guix pull --cores=1
>
>
>
[-- Attachment #2: Type: text/html, Size: 2205 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull
2022-07-13 17:21 ` Julien Lepiller
@ 2022-07-13 20:00 ` Liliana Marie Prikler
2022-07-14 12:50 ` Csepp
0 siblings, 1 reply; 6+ messages in thread
From: Liliana Marie Prikler @ 2022-07-13 20:00 UTC (permalink / raw)
To: Julien Lepiller, Csepp, Maxim Cournoyer; +Cc: dannym, 39813, 39813-done
Am Mittwoch, dem 13.07.2022 um 19:21 +0200 schrieb Julien Lepiller:
> I've heard that theory before. From observation on my late armhf
> server
> (two cores):
>
> - it takes just below 2GB to build one of the derivations
> - It doesn't swap a single byte
> - whether with two or a single core, it takes roughly the same amount
> of memory
> - substitution is nice, it doesn't require lots of memory (and then -
> -
> cores is useless)
>
> I think it's because we load all the files in a batch before we build
> them. The biggest amount of memory required is not for running the
> compiler on a thread, but for loading files and keeping them in
> memory for the whole duration of the build. With more threads, we
> still don't load each file more than once (twice to build it), so
> there's no reason it should take more memory.
>
> Or maybe the process of loading and building is inherently single-
> threaded? I don't think so, but maybe?
Loading and building is implemented in build-aux/compile-all.scm, which
does use multiple parallel workers. However, since all compilation
appears to be done in the same Guile process, I don't think multi-
threading makes too big of an impact (it'll be the same garbage
collector regardless of ordering).
Cheers
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull
2022-07-13 20:00 ` Liliana Marie Prikler
@ 2022-07-14 12:50 ` Csepp
0 siblings, 0 replies; 6+ messages in thread
From: Csepp @ 2022-07-14 12:50 UTC (permalink / raw)
To: Liliana Marie Prikler
Cc: Julien Lepiller, Maxim Cournoyer, dannym, 39813, 39813-done,
Csepp
Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> Am Mittwoch, dem 13.07.2022 um 19:21 +0200 schrieb Julien Lepiller:
>> I've heard that theory before. From observation on my late armhf
>> server
>> (two cores):
>>
>> - it takes just below 2GB to build one of the derivations
>> - It doesn't swap a single byte
>> - whether with two or a single core, it takes roughly the same amount
>> of memory
>> - substitution is nice, it doesn't require lots of memory (and then -
>> -
>> cores is useless)
>>
>> I think it's because we load all the files in a batch before we build
>> them. The biggest amount of memory required is not for running the
>> compiler on a thread, but for loading files and keeping them in
>> memory for the whole duration of the build. With more threads, we
>> still don't load each file more than once (twice to build it), so
>> there's no reason it should take more memory.
>>
>> Or maybe the process of loading and building is inherently single-
>> threaded? I don't think so, but maybe?
> Loading and building is implemented in build-aux/compile-all.scm, which
> does use multiple parallel workers. However, since all compilation
> appears to be done in the same Guile process, I don't think multi-
> threading makes too big of an impact (it'll be the same garbage
> collector regardless of ordering).
>
> Cheers
Hmm, for some reason it finishes much faster on my i686 netbook with
--cores=1. I'll try to look into it further. I enable swap on it but it
doesn't use a lot of it. Maybe using an HDD matters too, since swapping
is much more expensive on it. Disabling parallelism tends to help because
it can halve the worst case memory usage. If foo.c and bar.c both
require at most 1M during build, building them in parallel is 2M in the
worst case, but only 1M when serialized. Oh and this is with
channel-with-substitutes-available. So either that's broken or something
still needs building.
Anyways, I should take a looksie at build-aux/compile-all.scm, maybe I
can decrease the memory usage.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-07-14 12:58 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-27 20:06 bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull Danny Milosavljevic
2022-07-13 14:00 ` Maxim Cournoyer
2022-07-13 16:58 ` Csepp
2022-07-13 17:21 ` Julien Lepiller
2022-07-13 20:00 ` Liliana Marie Prikler
2022-07-14 12:50 ` Csepp
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).