* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull @ 2020-02-27 20:06 Danny Milosavljevic 2022-07-13 14:00 ` Maxim Cournoyer 0 siblings, 1 reply; 6+ messages in thread From: Danny Milosavljevic @ 2020-02-27 20:06 UTC (permalink / raw) To: 39813 [-- Attachment #1: Type: text/plain, Size: 244 bytes --] Hi, I just got a report that with Guix in a virtual Machine (like described in the manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM. It does work fine with 4 GB of RAM. [1] build .....guix-packages-base.drv failed [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull 2020-02-27 20:06 bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull Danny Milosavljevic @ 2022-07-13 14:00 ` Maxim Cournoyer 2022-07-13 16:58 ` Csepp 0 siblings, 1 reply; 6+ messages in thread From: Maxim Cournoyer @ 2022-07-13 14:00 UTC (permalink / raw) To: Danny Milosavljevic; +Cc: 39813-done Hi Danny, Danny Milosavljevic <dannym@scratchpost.org> writes: > Hi, > > I just got a report that with Guix in a virtual Machine (like described in the > manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM. > It does work fine with 4 GB of RAM. I don't see any reference of 1 GiB being enough in our current version of the manual. If you do, please let me know. Closing for now. Thanks, Maxim ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull 2022-07-13 14:00 ` Maxim Cournoyer @ 2022-07-13 16:58 ` Csepp 2022-07-13 17:21 ` Julien Lepiller 0 siblings, 1 reply; 6+ messages in thread From: Csepp @ 2022-07-13 16:58 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: dannym, 39813, 39813-done Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Hi Danny, > > Danny Milosavljevic <dannym@scratchpost.org> writes: > >> Hi, >> >> I just got a report that with Guix in a virtual Machine (like described in the >> manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM. >> It does work fine with 4 GB of RAM. > > I don't see any reference of 1 GiB being enough in our current version > of the manual. If you do, please let me know. > > Closing for now. > > Thanks, > > Maxim I think it's enough if you only use a single core. If any guix operations goes out of memory, add --cores=1. So: guix pull --cores=1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull 2022-07-13 16:58 ` Csepp @ 2022-07-13 17:21 ` Julien Lepiller 2022-07-13 20:00 ` Liliana Marie Prikler 0 siblings, 1 reply; 6+ messages in thread From: Julien Lepiller @ 2022-07-13 17:21 UTC (permalink / raw) To: 39813, raingloom, maxim.cournoyer; +Cc: dannym, 39813-done [-- Attachment #1: Type: text/plain, Size: 1645 bytes --] I've heard that theory before. From observation on my late armhf server (two cores): - it takes just below 2GB to build one of the derivations - It doesn't swap a single byte - whether with two or a single core, it takes roughly the same amount of memory - substitution is nice, it doesn't require lots of memory (and then --cores is useless) I think it's because we load all the files in a batch before we build them. The biggest amount of memory required is not for running the compiler on a thread, but for loading files and keeping them in memory for the whole duration of the build. With more threads, we still don't load each file more than once (twice to build it), so there's no reason it should take more memory. Or maybe the process of loading and building is inherently single-threaded? I don't think so, but maybe? Or it doesn't honor --cores. Le 13 juillet 2022 18:58:58 GMT+02:00, Csepp <raingloom@riseup.net> a écrit : > >Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > >> Hi Danny, >> >> Danny Milosavljevic <dannym@scratchpost.org> writes: >> >>> Hi, >>> >>> I just got a report that with Guix in a virtual Machine (like described in the >>> manual in 8.16), guix pull does not actually work[1] with 1 GB of RAM. >>> It does work fine with 4 GB of RAM. >> >> I don't see any reference of 1 GiB being enough in our current version >> of the manual. If you do, please let me know. >> >> Closing for now. >> >> Thanks, >> >> Maxim > >I think it's enough if you only use a single core. >If any guix operations goes out of memory, add --cores=1. >So: guix pull --cores=1 > > > [-- Attachment #2: Type: text/html, Size: 2205 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull 2022-07-13 17:21 ` Julien Lepiller @ 2022-07-13 20:00 ` Liliana Marie Prikler 2022-07-14 12:50 ` Csepp 0 siblings, 1 reply; 6+ messages in thread From: Liliana Marie Prikler @ 2022-07-13 20:00 UTC (permalink / raw) To: Julien Lepiller, Csepp, Maxim Cournoyer; +Cc: dannym, 39813, 39813-done Am Mittwoch, dem 13.07.2022 um 19:21 +0200 schrieb Julien Lepiller: > I've heard that theory before. From observation on my late armhf > server > (two cores): > > - it takes just below 2GB to build one of the derivations > - It doesn't swap a single byte > - whether with two or a single core, it takes roughly the same amount > of memory > - substitution is nice, it doesn't require lots of memory (and then - > - > cores is useless) > > I think it's because we load all the files in a batch before we build > them. The biggest amount of memory required is not for running the > compiler on a thread, but for loading files and keeping them in > memory for the whole duration of the build. With more threads, we > still don't load each file more than once (twice to build it), so > there's no reason it should take more memory. > > Or maybe the process of loading and building is inherently single- > threaded? I don't think so, but maybe? Loading and building is implemented in build-aux/compile-all.scm, which does use multiple parallel workers. However, since all compilation appears to be done in the same Guile process, I don't think multi- threading makes too big of an impact (it'll be the same garbage collector regardless of ordering). Cheers ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull 2022-07-13 20:00 ` Liliana Marie Prikler @ 2022-07-14 12:50 ` Csepp 0 siblings, 0 replies; 6+ messages in thread From: Csepp @ 2022-07-14 12:50 UTC (permalink / raw) To: Liliana Marie Prikler Cc: Julien Lepiller, Maxim Cournoyer, dannym, 39813, 39813-done, Csepp Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > Am Mittwoch, dem 13.07.2022 um 19:21 +0200 schrieb Julien Lepiller: >> I've heard that theory before. From observation on my late armhf >> server >> (two cores): >> >> - it takes just below 2GB to build one of the derivations >> - It doesn't swap a single byte >> - whether with two or a single core, it takes roughly the same amount >> of memory >> - substitution is nice, it doesn't require lots of memory (and then - >> - >> cores is useless) >> >> I think it's because we load all the files in a batch before we build >> them. The biggest amount of memory required is not for running the >> compiler on a thread, but for loading files and keeping them in >> memory for the whole duration of the build. With more threads, we >> still don't load each file more than once (twice to build it), so >> there's no reason it should take more memory. >> >> Or maybe the process of loading and building is inherently single- >> threaded? I don't think so, but maybe? > Loading and building is implemented in build-aux/compile-all.scm, which > does use multiple parallel workers. However, since all compilation > appears to be done in the same Guile process, I don't think multi- > threading makes too big of an impact (it'll be the same garbage > collector regardless of ordering). > > Cheers Hmm, for some reason it finishes much faster on my i686 netbook with --cores=1. I'll try to look into it further. I enable swap on it but it doesn't use a lot of it. Maybe using an HDD matters too, since swapping is much more expensive on it. Disabling parallelism tends to help because it can halve the worst case memory usage. If foo.c and bar.c both require at most 1M during build, building them in parallel is 2M in the worst case, but only 1M when serialized. Oh and this is with channel-with-substitutes-available. So either that's broken or something still needs building. Anyways, I should take a looksie at build-aux/compile-all.scm, maybe I can decrease the memory usage. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-07-14 12:58 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-02-27 20:06 bug#39813: Running Guix in a Virtual Machine - says 1 GB RAM is enough, but it isn't for guix pull Danny Milosavljevic 2022-07-13 14:00 ` Maxim Cournoyer 2022-07-13 16:58 ` Csepp 2022-07-13 17:21 ` Julien Lepiller 2022-07-13 20:00 ` Liliana Marie Prikler 2022-07-14 12:50 ` Csepp
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).