* bug#42371: Grafts max out build slots without being queued
@ 2020-07-15 16:13 Maxim Cournoyer
2020-07-15 17:13 ` Maxim Cournoyer
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2020-07-15 16:13 UTC (permalink / raw)
To: 42371
Hello,
While working on ruby-asciidoctor-pdf, the following command failed
because of maxing out build users:
./pre-inst-env guix build --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1
The following derivations will be built:
/gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv
/gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv
/gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv
/gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv
/gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv
/gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv
/gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv
/gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv
/gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv
/gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv
The following grafts will be made:
/gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv
/gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv
/gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv
/gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv
/gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv
/gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv
/gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv
/gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv
/gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv
/gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv
/gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv
guix build: error: all build users are currently in use; consider
creating additional users and adding them to the `guixbuild' group
It seems that grafts trigger the problem, as passing the --no-grafts
option works around it. Could it be that grafts can't somehow be queued
like normal builds, adapting to fit to the available build slots?
Maxim
^ permalink raw reply [flat|nested] 11+ messages in thread* bug#42371: Grafts max out build slots without being queued 2020-07-15 16:13 bug#42371: Grafts max out build slots without being queued Maxim Cournoyer @ 2020-07-15 17:13 ` Maxim Cournoyer 2020-07-16 10:18 ` Ludovic Courtès ` (2 subsequent siblings) 3 siblings, 0 replies; 11+ messages in thread From: Maxim Cournoyer @ 2020-07-15 17:13 UTC (permalink / raw) To: 42371 Hello, Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > It seems that grafts trigger the problem, as passing the --no-grafts > option works around it. Could it be that grafts can't somehow be queued > like normal builds, adapting to fit to the available build slots? I sent this message too early; the '--no-grafts' option does *not* work around the issue... I wonder what triggers this here? I commonly build things (sometimes hundreds of packages at the same time) and never ran into this issue. Maxim ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-15 16:13 bug#42371: Grafts max out build slots without being queued Maxim Cournoyer 2020-07-15 17:13 ` Maxim Cournoyer @ 2020-07-16 10:18 ` Ludovic Courtès 2020-07-16 11:09 ` zimoun 2020-07-16 22:06 ` Maxim Cournoyer 2020-07-16 11:04 ` zimoun 2020-12-04 14:18 ` zimoun 3 siblings, 2 replies; 11+ messages in thread From: Ludovic Courtès @ 2020-07-16 10:18 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 42371 Hi, Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis: > While working on ruby-asciidoctor-pdf, the following command failed > because of maxing out build users: > > ./pre-inst-env guix build --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1 > The following derivations will be built: > /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv > /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv > /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv > /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv > /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv > /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv > /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv > /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv > /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv > /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv > The following grafts will be made: > /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv > /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv > /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv > /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv > /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv > /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv > /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv > /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv > /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv > /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv > /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv > guix build: error: all build users are currently in use; consider > creating additional users and adding them to the `guixbuild' group Presumably here it’s running one job at a time, so I don’t think it can end up using more than one build user. What does “guix processes” return when that happens? Could it be that some other Guix clients are running? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-16 10:18 ` Ludovic Courtès @ 2020-07-16 11:09 ` zimoun 2020-07-16 22:06 ` Maxim Cournoyer 1 sibling, 0 replies; 11+ messages in thread From: zimoun @ 2020-07-16 11:09 UTC (permalink / raw) To: Ludovic Courtès, Maxim Cournoyer; +Cc: 42371 On Thu, 16 Jul 2020 at 12:18, Ludovic Courtès <ludo@gnu.org> wrote: > Presumably here it’s running one job at a time, so I don’t think it can > end up using more than one build user. > > What does “guix processes” return when that happens? Could it be that > some other Guix clients are running? Is it possible that the "lockFile()" is busy for another reason? Cheers, simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-16 10:18 ` Ludovic Courtès 2020-07-16 11:09 ` zimoun @ 2020-07-16 22:06 ` Maxim Cournoyer 1 sibling, 0 replies; 11+ messages in thread From: Maxim Cournoyer @ 2020-07-16 22:06 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 42371 Hi Ludo :-) Ludovic Courtès <ludo@gnu.org> writes: > Hi, > > Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis: > >> While working on ruby-asciidoctor-pdf, the following command failed >> because of maxing out build users: >> >> ./pre-inst-env guix build --quiet --keep-going --check >> ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 >> numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 >> ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 >> ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 >> warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 >> ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 >> ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 >> ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1 >> The following derivations will be built: >> /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv >> /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv >> /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv >> /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv >> /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv >> /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv >> /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv >> /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv >> /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv >> /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv >> The following grafts will be made: >> /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv >> /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv >> /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv >> /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv >> /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv >> /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv >> /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv >> /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv >> /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv >> /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv >> /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv >> guix build: error: all build users are currently in use; consider >> creating additional users and adding them to the `guixbuild' group > > Presumably here it’s running one job at a time, so I don’t think it can > end up using more than one build user. > > What does “guix processes” return when that happens? Could it be that > some other Guix clients are running? > > Thanks, > Ludo’. So, I'm afraid this isn't very useful, but: for the most part of the process, I had something like: --8<---------------cut here---------------start------------->8--- SessionPID: 12549 ClientPID: 12540 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment --pure gcc-toolchain@10 --ad-hoc coreutils strace the-silver-searcher findutils SessionPID: 1876 ClientPID: 1868 ClientCommand: /gnu/store/6ms9q743slbyx4zj7p97sqrwx8x3dgkm-profile/bin/guile \ /home/maxim/src/guix/scripts/guix build --no-grafts --max-jobs=1 --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 SessionPID: 447 ClientPID: 441 ClientCommand: /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2/bin/guile \ /gnu/store/3f4xln2mii6cmc4c6x9138wl0bsh0fs7-guix-1.1.0-7.d445c30/bin/.guix-real publish -u guix-publish -p 80 -C gzip:3 --nar-path=nar --listen=0.0.0.0 SessionPID: 7060 ClientPID: 7039 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment guix --8<---------------cut here---------------end--------------->8--- And after the same error was printed (guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group), then I had: --8<---------------cut here---------------start------------->8--- SessionPID: 12549 ClientPID: 12540 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment --pure gcc-toolchain@10 --ad-hoc coreutils strace the-silver-searcher findutils SessionPID: 447 ClientPID: 441 ClientCommand: /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2/bin/guile \ /gnu/store/3f4xln2mii6cmc4c6x9138wl0bsh0fs7-guix-1.1.0-7.d445c30/bin/.guix-real publish -u guix-publish -p 80 -C gzip:3 --nar-path=nar --listen=0.0.0.0 SessionPID: 7060 ClientPID: 7039 ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment guix --8<---------------cut here---------------end--------------->8--- I didn't see the amount of build process going higher than than, monitoring it every second. I've pushed the branch I'm using to test locally here [0], hoping it'll help to reproduce the issue somewhere else. The command to run on this branch is: ./pre-inst-env guix build --no-grafts --max-jobs=1 --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 The bug is only triggered when I also pass the "--check" option. So you'll probably have to build the stack once without "--check", then a second time with "--check" passed to trigger the issue (if it's reproducible). Thank you! Maxim [0] https://gitlab.com/Apteryks/guix/-/tree/add-ruby-asciidoctor-pdf ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-15 16:13 bug#42371: Grafts max out build slots without being queued Maxim Cournoyer 2020-07-15 17:13 ` Maxim Cournoyer 2020-07-16 10:18 ` Ludovic Courtès @ 2020-07-16 11:04 ` zimoun 2020-07-17 2:38 ` Maxim Cournoyer 2020-07-17 3:06 ` Maxim Cournoyer 2020-12-04 14:18 ` zimoun 3 siblings, 2 replies; 11+ messages in thread From: zimoun @ 2020-07-16 11:04 UTC (permalink / raw) To: Maxim Cournoyer, 42371 Dear Maxim, On Wed, 15 Jul 2020 at 12:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote: > ./pre-inst-env guix build --quiet --keep-going --check > ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 > numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 > ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 > ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 > warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 > ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 > ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 > ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1 Instead, I have run: --8<---------------cut here---------------start------------->8--- guix build --keep-going --check ruby_version gnome-shell-extension-gsconnect numix-gtk-theme arc-theme gnome ruby-fakefs git-minimal ruby-fast-gettext ruby-sass-rails mailcatcher ruby-rspec-its warzone2100 keepassxc wsjtx js8call ruby-web-console hello ruby-autoprefixer-rails ruby-jekyll-paginate-v2 ruby-solargraph --max-jobs=1 --8<---------------cut here---------------end--------------->8--- replacing the 2 unknown packages ruby-yard-with-tests and ruby-protobuf by git-minimal and hello. And it passes. I have 10 guixbuilder users, as it is by default. :-) > The following derivations will be built: > /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv > /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv > /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv > /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv > /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv > /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv > /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv > /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv > /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv > /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv > The following grafts will be made: > /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv > /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv > /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv > /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv > /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv > /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv > /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv > /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv > /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv > /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv > /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv > guix build: error: all build users are currently in use; consider > creating additional users and adding them to the `guixbuild' group How many users do you have? What are they doing? All the best, simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-16 11:04 ` zimoun @ 2020-07-17 2:38 ` Maxim Cournoyer 2020-07-17 3:06 ` Maxim Cournoyer 1 sibling, 0 replies; 11+ messages in thread From: Maxim Cournoyer @ 2020-07-17 2:38 UTC (permalink / raw) To: zimoun; +Cc: 42371 Hey Simon, zimoun <zimon.toutoune@gmail.com> writes: > Dear Maxim, > > > On Wed, 15 Jul 2020 at 12:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote: > >> ./pre-inst-env guix build --quiet --keep-going --check >> ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 >> numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 >> ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 >> ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 >> warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 >> ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 >> ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 >> ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1 > > Instead, I have run: > > guix build --keep-going --check ruby_version > gnome-shell-extension-gsconnect numix-gtk-theme arc-theme gnome > ruby-fakefs git-minimal ruby-fast-gettext ruby-sass-rails mailcatcher > ruby-rspec-its warzone2100 keepassxc wsjtx js8call ruby-web-console > hello ruby-autoprefixer-rails ruby-jekyll-paginate-v2 ruby-solargraph > --max-jobs=1 > > replacing the 2 unknown packages ruby-yard-with-tests and ruby-protobuf > by git-minimal and hello. Thank you for your tests! This makes me wonder whether these are the ones causing the problem... I've uploaded the branch I'm working on so that you could try to reproduce [0] if you are so inclined :-). > [...] I have 10 guixbuilder users, as it is by default. :-) Same here. [...] >> guix build: error: all build users are currently in use; consider >> creating additional users and adding them to the `guixbuild' group > > How many users do you have? What are they doing? 10 users, not doing anything special, as I found out looking at guix processes, so I don't know what is causing the exhaustion of build users. Thanks, Maxim [0] https://gitlab.com/Apteryks/guix/-/tree/add-ruby-asciidoctor-pdf ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-16 11:04 ` zimoun 2020-07-17 2:38 ` Maxim Cournoyer @ 2020-07-17 3:06 ` Maxim Cournoyer 2020-07-17 9:23 ` zimoun 2020-07-21 10:40 ` zimoun 1 sibling, 2 replies; 11+ messages in thread From: Maxim Cournoyer @ 2020-07-17 3:06 UTC (permalink / raw) To: zimoun; +Cc: 42371 Hello everyone, I did a couple more tests, and there's no need to get fancy with the reproduction steps; any number greater of packages greater than your number of build slots/users should suffice: --8<---------------cut here---------------start------------->8--- guix build --quiet --check $(guix package -A | head -n 11 | cut -f1) The following derivations will be built: /gnu/store/l7hrbi8cikf4c0calvqwd2n99v51ljzb-aalib-1.4rc5.drv /gnu/store/bijqg7w67rq1lr20hqbk18rq73963xfr-a2ps-4.14.drv /gnu/store/vh283nmxk08sl0m89s1fn07m432c9dg0-4ti2-1.6.9.drv /gnu/store/davmjim4nfvbhimpim329p642p9mg50d-0xffff-0.8.drv /gnu/store/7qaf91fmf42c8vp3fcysvqpkg4zahl1j-0ad-data-0.0.23b-alpha.drv The following grafts will be made: /gnu/store/hjr33zqgx46ldswbqlmvgz7g37l7hrxl-abbaye-2.0.1.drv /gnu/store/71rdqs3hldmy1d92742camwc97l3pb3w-7kaa-2.15.3.drv /gnu/store/0b92jff2kfmd6l23y8v9xcprk8l7casw-4store-1.1.6.drv /gnu/store/82pj69v420fjmkqjxjxix3cckwcfdwg2-4dtris-0.4.3.drv /gnu/store/gamc676fdg1p7qzvzwfq7rp6qn4agss1-389-ds-base-1.4.0.21.drv /gnu/store/p310ysfwix0ci2594x78vv4x5isqcxix-0ad-0.0.23b-alpha.drv guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group --8<---------------cut here---------------end--------------->8--- The key is passing more than 10 packages (I have 10 build users) as an argument when using the '--check' argument. I'll update the title to reflect the actual bug. Maxim ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-17 3:06 ` Maxim Cournoyer @ 2020-07-17 9:23 ` zimoun 2020-07-21 10:40 ` zimoun 1 sibling, 0 replies; 11+ messages in thread From: zimoun @ 2020-07-17 9:23 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 42371 Hi Maxim, > --8<---------------cut here---------------start------------->8--- > guix build --quiet --check $(guix package -A | head -n 11 | cut -f1) [...] > guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group > --8<---------------cut here---------------end--------------->8--- In my case, it starts to download substitutes (but I have remove --quiet), something like: --8<---------------cut here---------------start------------->8--- substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% 11.3 MB will be downloaded: /gnu/store/…-jansson-2.12 /gnu/store/…-jemalloc-5.2.1 /gnu/store/…-libev-4.31 /gnu/store/…-glibc-2.31 /gnu/store/…-glibc-2.31-static /gnu/store/…-c-ares-1.16.0 /gnu/store/…-nghttp2-1.41.0-lib /gnu/store/…-nghttp2-1.41.0 substituting /gnu/store/…-c-ares-1.16.0... downloading from https://ci.guix.gnu.org/nar/lzip/84hjhzjypgp5pgh6r5y7m561lk625\ pn0-c-ares-1.16.0 ... c-ares-1.16.0 123KiB 1.6MiB/s 00:00 [##################] 100.0% substituting /gnu/store/…-jansson-2.12... downloading from https://ci.guix.gnu.org/nar/lzip/v5rgf8v6cjxjbngkzjcznj98dkxj8\ svg-jansson-2.12 ... jansson-2.12 27KiB 4.3MiB/s 00:00 [##################] 100.0% […] 13.2 MB will be downloaded: /gnu/store/…-cyrus-sasl-2.1.27 /gnu/store/…-glibc-2.31 /gnu/store/…-glibc-2.31-static /gnu/store/…-gnutls-3.6.12 /gnu/store/…-openldap-2.4.50 substituting /gnu/store/…-cyrus-sasl-2.1.27... downloading from https://ci.guix.gnu.org/nar/lzip/xp8545c74xkkj12q41d1zl5wngb47\ 7gj-cyrus-sasl-2.1.27 ... cyrus-sasl-2.1.27 234K […] 28.0 MB will be downloaded: /gnu/store/…-pcre-8.44-bin /gnu/store/…-pcre-8.44 /gnu/store/…-mpfr-4.0.2 /gnu/store/…-rasqal-0.9.33 /gnu/store/…-libxslt-1.1.34 /gnu/store/…-raptor2-2.0.15 /gnu/store/…-glib-2.62.6 /gnu/store/…-dbus-1.12.16 /gnu/store/…-glib-2.62.6-bin /gnu/store/…-glib-2.62.6 /gnu/store/…-avahi-0.8 /gnu/store/…-4store-1.1.6 /gnu/store/…-a2ps-4.14 /gnu/store/…-mesa-20.0.8-bin /gnu/store/…-mesa-20.0.8 […] The following derivations will be built: /gnu/store/…-abbaye-2.0.1.drv /gnu/store/…-sdl-union-1.2.15.drv /gnu/store/…-0ad-0.0.23b-alpha.drv 899.1 MB will be downloaded: /gnu/store/…-libsodium-1.0.18 /gnu/store/…-mozjs-38.2.1.rc0 […] /gnu/store/…-ed-1.16 /gnu/store/…-patch-2.7.6 /gnu/store/…-diffutils-3.7 […] successfully built /gnu/store/…-sdl-union-1.2.15.drv building /gnu/store/…-0ad-0.0.23b-alpha.drv... WARNING: (guile-user): imported module (guix build utils) overrides core bindin\ g `delete' starting phase `set-SOURCE-DATE-EPOCH' phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds starting phase `set-paths' environment variable `PATH' … […] --8<---------------cut here---------------end--------------->8--- And I even tried with "-n21" and it seems ok. I am running Guix on the top of Debian with ext4. > The key is passing more than 10 packages (I have 10 build users) as an > argument when using the '--check' argument. Well, I am not sure to understand, but in nix/libstore/build.cc, "void UserLock::acquire()", there is: --8<---------------cut here---------------start------------->8--- foreach (Strings::iterator, i, users) { […] if (lockFile(fd, ltWrite, false)) { […] return; } } throw Error(format("all build users are currently in use; " "consider creating additional users and adding them to the `%1%' group") % settings.buildUsersGroup); --8<---------------cut here---------------end--------------->8--- So maybe the file is locked for whatever other reason on your machine? All the best, simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-17 3:06 ` Maxim Cournoyer 2020-07-17 9:23 ` zimoun @ 2020-07-21 10:40 ` zimoun 1 sibling, 0 replies; 11+ messages in thread From: zimoun @ 2020-07-21 10:40 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 42371 Dear, On my laptop and my desktop, both “cat /proc/cpuinfo |grep processor|wc -l” returning 4, and with 10 guixbuild users, I get these results. On Thu, 16 Jul 2020 at 23:06, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote: > --8<---------------cut here---------------start------------->8--- > guix build --quiet --check $(guix package -A | head -n 11 | cut -f1) […] guix build: error: derivation `/gnu/store/…-4ti2-1.6.9.drv' may not be determin\ istic: output `/gnu/store/…-4ti2-1.6.9' differs > --8<---------------cut here---------------end--------------->8--- Then, I run this tiny script fixing --max-jobs and --cores on my desktop machine only: --8<---------------cut here---------------start------------->8--- for npkg in 4 5 6 7 8 9 10 11 12 do for JOBS in 1 2 3 4 do for CORES in 1 2 4 8 16 32 do echo "# num.of pkg=$npkg" echo "# --max-jobs=$JOBS" echo "# --cores=$CORES" guix build --quiet --check -k \ --max-jobs=$JOBS \ --cores=$CORES \ $(guix package -A | head -n $npkg | cut -f1) echo " " sleep 2 done done done --8<---------------cut here---------------end--------------->8--- And it hits the non-deterministic error for all the cases until the case num.of pkg=11 --max-jobs=2 --cores=1 where it errors "all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group". Then npkg=11 --max-jobs=2 --cores=2 fails with non-deterministic. (I reproduce on my laptop the result for npkg=11 max-jobs=2 cores=1,2.) The next case, npkg=11 --max-jobs=2 --cores=4 does not seem reproducible on my machine; similarly for cores=8,16,32. Then I have not checked the reproducibility of the cases below. Interestingly, npkg=11 --max-jobs=3 --cores=1,2,4,8,16 fails with not enough builders. But not for --cores=32 which fails with non-deterministic. All the values for cores fails with not enough builder for npkg=11 and --max-jobs=4. Another surprise, npkg=12 --max-jobs=1 --cores=1,2,4,8,16,32 fails with non-deterministic. And npkg=12 --max-jobs=2 --cores=2,4,8,16,32 too. All the best, simon ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#42371: Grafts max out build slots without being queued 2020-07-15 16:13 bug#42371: Grafts max out build slots without being queued Maxim Cournoyer ` (2 preceding siblings ...) 2020-07-16 11:04 ` zimoun @ 2020-12-04 14:18 ` zimoun 3 siblings, 0 replies; 11+ messages in thread From: zimoun @ 2020-12-04 14:18 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 42371 Hi, As a reminder about the annoyance of the bug, I get: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build -m ~/tmp/Guix/list-bioconductor.scm -k -v0 --check guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group --8<---------------cut here---------------end--------------->8--- where list-bioconductor.scm is below. It is annoying because the bug prevents massive rebuild; for instance here the ones from the Bioconductor archive, useful when massive update or to track reproducibility issue. All the best, simon --8<---------------cut here---------------start------------->8--- (use-modules (guix) (gnu) (guix download) (guix svn-download) (guix git-download) (ice-9 match) (srfi srfi-1) (srfi srfi-26)) (define (bioconductor? package) (define (bioc-string? str) (string-contains str "bioconductor.org")) (match (package-source package) ((? origin? o) (match (origin-uri o) ((? string? url) (bioc-string? url)) (((? string? urls) ...) (any bioc-string? urls)) ;or 'find' (_ #f))) (_ #f))) (define packages-from-bioconductor (fold-packages (lambda (package result) (if (bioconductor? package) (cons package result) result)) '())) ;; (map (lambda (pkg) ;; (begin ;; (format #t "~A~%"(package-name pkg)))) ;; packages-from-bioconductor) (specifications->manifest (map package-name packages-from-bioconductor)) --8<---------------cut here---------------end--------------->8--- ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-12-08 18:44 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-07-15 16:13 bug#42371: Grafts max out build slots without being queued Maxim Cournoyer 2020-07-15 17:13 ` Maxim Cournoyer 2020-07-16 10:18 ` Ludovic Courtès 2020-07-16 11:09 ` zimoun 2020-07-16 22:06 ` Maxim Cournoyer 2020-07-16 11:04 ` zimoun 2020-07-17 2:38 ` Maxim Cournoyer 2020-07-17 3:06 ` Maxim Cournoyer 2020-07-17 9:23 ` zimoun 2020-07-21 10:40 ` zimoun 2020-12-04 14:18 ` zimoun
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).