* Integration of the Nix daemon
@ 2012-12-03 22:13 Ludovic Courtès
2012-12-13 21:17 ` Ludovic Courtès
0 siblings, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-03 22:13 UTC (permalink / raw
To: bug-guix
Hi,
I just pushed a new ‘nix-integration’ branch. The goal is to import the
subset of Nix that we need to run the daemon in Guix itself, so that 0.1
will hopefully be self-contained.
The branch adds a script to synchronize the code with upstream Nix
(think ‘gnulib-update’.) It extends the build system to build that C++
code, and adds a little bit of glue code to put it all together.
There’s still some functionality missing. In Nix it’s provided by a
bunch of hairy Perl scripts that nix-daemon calls, but I’d like to
rework that part in an attempt to honor a zero-Perl policy. :-)
Comments welcome!
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-03 22:13 Integration of the Nix daemon Ludovic Courtès
@ 2012-12-13 21:17 ` Ludovic Courtès
2012-12-13 22:06 ` Andreas Enge
2012-12-13 23:17 ` Andreas Enge
0 siblings, 2 replies; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-13 21:17 UTC (permalink / raw
To: bug-guix
Hello!
I just merged that branch in ‘master’. So now, you need to run
‘./bootstrap’ (and not just autoreconf) to initialize the package.
So now, unless ‘--disable-daemon’ is passed, a ‘guix-daemon’ binary is
built from C++, mostly taken from upstream Nix.
Please report any problems. :-)
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 21:17 ` Ludovic Courtès
@ 2012-12-13 22:06 ` Andreas Enge
2012-12-13 22:56 ` Ludovic Courtès
2012-12-13 23:17 ` Andreas Enge
1 sibling, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-13 22:06 UTC (permalink / raw
To: bug-guix
[-- Attachment #1: Type: text/plain, Size: 427 bytes --]
Am Donnerstag, 13. Dezember 2012 schrieb Ludovic Courtès:
> So now, unless ‘--disable-daemon’ is passed, a ‘guix-daemon’ binary is
> built from C++, mostly taken from upstream Nix.
I also tried "--disable-daemon". It does not make and install the daemon,
but guix-package still tries to communicate with a socket in the
installation directory. So it does not cooperate with the nix-worker
anymore.
Andreas
[-- Attachment #2: Type: text/html, Size: 2369 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 22:06 ` Andreas Enge
@ 2012-12-13 22:56 ` Ludovic Courtès
2012-12-13 23:10 ` Andreas Enge
0 siblings, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-13 22:56 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Andreas Enge <andreas@enge.fr> skribis:
> Am Donnerstag, 13. Dezember 2012 schrieb Ludovic Courtès:
>> So now, unless ‘--disable-daemon’ is passed, a ‘guix-daemon’ binary is
>> built from C++, mostly taken from upstream Nix.
>
> I also tried "--disable-daemon". It does not make and install the daemon,
> but guix-package still tries to communicate with a socket in the
> installation directory. So it does not cooperate with the nix-worker
> anymore.
Yes: in that case, you should use --localstatedir to specify the place
where nix-worker has its data and socket.
I guess README should mention it.
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 22:56 ` Ludovic Courtès
@ 2012-12-13 23:10 ` Andreas Enge
2013-01-15 23:03 ` Ludovic Courtès
0 siblings, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-13 23:10 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1: Type: text/plain, Size: 383 bytes --]
Am Donnerstag, 13. Dezember 2012 schrieb Ludovic Courtès:
> Yes: in that case, you should use --localstatedir to specify the place
> where nix-worker has its data and socket.
>
> I guess README should mention it.
As well as
guix-package --help
One more reason to not have guix-package connect to the daemon when the
options --help or --version are passed.
Andreas
[-- Attachment #2: Type: text/html, Size: 2603 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 21:17 ` Ludovic Courtès
2012-12-13 22:06 ` Andreas Enge
@ 2012-12-13 23:17 ` Andreas Enge
2012-12-14 10:07 ` Andreas Enge
2012-12-14 10:12 ` Ludovic Courtès
1 sibling, 2 replies; 25+ messages in thread
From: Andreas Enge @ 2012-12-13 23:17 UTC (permalink / raw
To: bug-guix
[-- Attachment #1: Type: text/plain, Size: 465 bytes --]
The command
./pre-inst-env guix-build hello
fails with
error: build failed: substituter `/usr/local/guix-
git/libexec/nix/substituters/download-using-manifests.pl' failed: error:
executing `/usr/local/guix-git/libexec/nix/substituters/download-using-
manifests.pl': No such file or directory
We have by
find /usr/local/guix-git/libexec
/usr/local/guix-git/libexec
/usr/local/guix-git/libexec/guix
/usr/local/guix-git/libexec/guix/list-runtime-roots
Andreas
[-- Attachment #2: Type: text/html, Size: 2976 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 23:17 ` Andreas Enge
@ 2012-12-14 10:07 ` Andreas Enge
2012-12-14 23:07 ` Ludovic Courtès
2012-12-14 10:12 ` Ludovic Courtès
1 sibling, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-14 10:07 UTC (permalink / raw
To: bug-guix
[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]
Am Freitag, 14. Dezember 2012 schrieb Andreas Enge:
> The command
> ./pre-inst-env guix-build hello
> fails with
>
> error: build failed: substituter `/usr/local/guix-
> git/libexec/nix/substituters/download-using-manifests.pl' failed: error:
> executing `/usr/local/guix-git/libexec/nix/substituters/download-using-
> manifests.pl': No such file or directory
As for nix-worker, it helps to call
NIX_SUBSTITUTERS="" guix-daemon --build-users-group=nixbld
Could something be done globally about this?
The next error message is
@ build-started /nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-
import.drv /nix/store/fdzwc7ss7zbpg60n5h2kpp98l8sbjx9f-module-import
x86_64-linux /usr/local/guix-
git/var/log/nix/drvs/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-import.drv.bz2
error: build failed: all build users are currently in use; consider
creating additional users and adding them to the `nixbld' group
although I just started the guix daemon and there are 10 users in the
group nixbld.
Andreas
[-- Attachment #2: Type: text/html, Size: 4679 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 23:17 ` Andreas Enge
2012-12-14 10:07 ` Andreas Enge
@ 2012-12-14 10:12 ` Ludovic Courtès
1 sibling, 0 replies; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-14 10:12 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Andreas Enge <andreas@enge.fr> skribis:
> The command
> ./pre-inst-env guix-build hello
> fails with
>
> error: build failed: substituter `/usr/local/guix-
> git/libexec/nix/substituters/download-using-manifests.pl' failed: error:
> executing `/usr/local/guix-git/libexec/nix/substituters/download-using-
> manifests.pl': No such file or directory
Ouch, right.
For now, ‘guix-daemon’ has to be run like this:
NIX_SUBSTITUTERS="" guix-daemon &
I’ll change it to be the default, so the env. var. doesn’t need to be set.
That’s because I didn’t want to import Nix’s substituters (scripts that
download pre-compiled binaries), mostly because they’re written in Perl.
We need a replacement, of course, but perhaps after 0.1 is out.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-14 10:07 ` Andreas Enge
@ 2012-12-14 23:07 ` Ludovic Courtès
2012-12-14 23:23 ` Andreas Enge
0 siblings, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-14 23:07 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Hi!
Andreas Enge <andreas@enge.fr> skribis:
> Am Freitag, 14. Dezember 2012 schrieb Andreas Enge:
>> The command
>> ./pre-inst-env guix-build hello
>> fails with
>>
>> error: build failed: substituter `/usr/local/guix-
>> git/libexec/nix/substituters/download-using-manifests.pl' failed: error:
>> executing `/usr/local/guix-git/libexec/nix/substituters/download-using-
>> manifests.pl': No such file or directory
>
> As for nix-worker, it helps to call
> NIX_SUBSTITUTERS="" guix-daemon --build-users-group=nixbld
> Could something be done globally about this?
Commit 335dd76 disables substituters. We’ll re-enable them once it’s
actually (re)implemented. :-)
> The next error message is
> @ build-started /nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-
> import.drv /nix/store/fdzwc7ss7zbpg60n5h2kpp98l8sbjx9f-module-import
> x86_64-linux /usr/local/guix-
> git/var/log/nix/drvs/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-import.drv.bz2
> error: build failed: all build users are currently in use; consider
> creating additional users and adding them to the `nixbld' group
>
> although I just started the guix daemon and there are 10 users in the
> group nixbld.
Hmm, can you try running guix-daemon with the newly-added ‘--debug’
option?
TIA,
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-14 23:07 ` Ludovic Courtès
@ 2012-12-14 23:23 ` Andreas Enge
2012-12-14 23:29 ` Andreas Enge
2012-12-15 14:26 ` Ludovic Courtès
0 siblings, 2 replies; 25+ messages in thread
From: Andreas Enge @ 2012-12-14 23:23 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1.1: Type: text/plain, Size: 1826 bytes --]
Am Samstag, 15. Dezember 2012 schrieb Ludovic Courtès:
> Commit 335dd76 disables substituters. We’ll re-enable them once it’s
> actually (re)implemented. :-)
Nice!
> > The next error message is
> > @ build-started /nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-
> > import.drv /nix/store/fdzwc7ss7zbpg60n5h2kpp98l8sbjx9f-module-import
> > x86_64-linux /usr/local/guix-
> > git/var/log/nix/drvs/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-import.dr
> > v.bz2 error: build failed: all build users are currently in use;
> > consider creating additional users and adding them to the `nixbld'
> > group
> > although I just started the guix daemon and there are 10 users in the
> > group nixbld.
> Hmm, can you try running guix-daemon with the newly-added ‘--debug’
> option?
Not much to see. Here is the output:
acquiring global GC lock `/usr/local/guix-git/var/nix/gc.lock'
acquiring read lock on `/usr/local/guix-git/var/nix/temproots/29720'
acquiring write lock on `/usr/local/guix-git/var/nix/temproots/29720'
downgrading to read lock on `/usr/local/guix-git/var/nix/temproots/29720'
... (the last two lines 1918 times)
the following derivations will be built:
/nix/store/rqllv82x33mfqvma7zbj2kjdkxdqdwpx-hello-2.8.drv
...
@ build-started /nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-
import.drv /nix/store/fdzwc7ss7zbpg60n5h2kpp98l8sbjx9f-module-import
x86_64-linux /usr/local/guix-
git/var/log/nix/drvs/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-import.drv.bz2
error: build failed: all build users are currently in use; consider
creating additional users and adding them to the `nixbld' group
I deleted /usr/local/guix-git/var/ and tried again
./pre-inst-env guix-build hello
The result is more varied, I am attaching it in case it is of any use.
Andreas
[-- Attachment #1.2: Type: text/html, Size: 7679 bytes --]
[-- Attachment #2: output.bz2 --]
[-- Type: application/x-bzip, Size: 15087 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-14 23:23 ` Andreas Enge
@ 2012-12-14 23:29 ` Andreas Enge
2012-12-15 14:48 ` Ludovic Courtès
2012-12-15 14:26 ` Ludovic Courtès
1 sibling, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-14 23:29 UTC (permalink / raw
To: bug-guix
[-- Attachment #1: Type: text/plain, Size: 196 bytes --]
One more comment: The size of guix-daemon is 9 804 696, that of
nix-worker 691 834, and the entire directory nix/bin contains
only 5 312 kB. A bit suspicious. Do you have an explanation?
Andreas
[-- Attachment #2: Type: text/html, Size: 1399 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-14 23:23 ` Andreas Enge
2012-12-14 23:29 ` Andreas Enge
@ 2012-12-15 14:26 ` Ludovic Courtès
2012-12-15 20:25 ` Andreas Enge
1 sibling, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-15 14:26 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Hi!
Andreas Enge <andreas@enge.fr> skribis:
> Not much to see.
Oh right. Actually, can you instead try this:
guix-build --verbosity=4 hello
(Yes, this is a brand new option. :-))
On my machine ‘guix-build’ emits messages like:
| | | found build user `nixbld1'
| | | found build user `nixbld2'
| | | found build user `nixbld3'
| | | found build user `nixbld4'
| | | found build user `nixbld5'
| | | found build user `nixbld6'
| | | found build user `nixbld7'
| | | found build user `nixbld8'
| | | found build user `nixbld9'
| | | found build user `nixbld10'
| | | trying user `nixbld1'
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-14 23:29 ` Andreas Enge
@ 2012-12-15 14:48 ` Ludovic Courtès
2012-12-15 19:38 ` Andreas Enge
0 siblings, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-15 14:48 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Andreas Enge <andreas@enge.fr> skribis:
> One more comment: The size of guix-daemon is 9 804 696, that of
> nix-worker 691 834, and the entire directory nix/bin contains
> only 5 312 kB. A bit suspicious. Do you have an explanation?
Running ‘strip --strip-debug guix-daemon’ makes it go from 10 MiB to
just 1 MiB (this is C++!).
Also, guix-daemon is statically-linked against the Nix libraries,
whereas nix-daemon is dynamically linked against them. Here,
/path/to/nix/lib/nix weighs in at around 2 MiB.
Does that answer your question?
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-15 14:48 ` Ludovic Courtès
@ 2012-12-15 19:38 ` Andreas Enge
0 siblings, 0 replies; 25+ messages in thread
From: Andreas Enge @ 2012-12-15 19:38 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1: Type: text/plain, Size: 486 bytes --]
Am Samstag, 15. Dezember 2012 schrieb Ludovic Courtès:
> Running ‘strip --strip-debug guix-daemon’ makes it go from 10 MiB to
> just 1 MiB (this is C++!).
>
> Also, guix-daemon is statically-linked against the Nix libraries,
> whereas nix-daemon is dynamically linked against them. Here,
> /path/to/nix/lib/nix weighs in at around 2 MiB.
>
> Does that answer your question?
Okay, very well. I saw that both were dynamically linked and started
to wonder.
Andreas
[-- Attachment #2: Type: text/html, Size: 3000 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-15 14:26 ` Ludovic Courtès
@ 2012-12-15 20:25 ` Andreas Enge
2012-12-15 22:59 ` Ludovic Courtès
0 siblings, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-15 20:25 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1.1: Type: text/plain, Size: 4210 bytes --]
Am Samstag, 15. Dezember 2012 schrieb Ludovic Courtès:
> Oh right. Actually, can you instead try this:
> guix-build --verbosity=4 hello
> (Yes, this is a brand new option. :-))
> On my machine ‘guix-build’ emits messages like:
> | | | found build user `nixbld1'
> | | | found build user `nixbld2'
> | | | found build user `nixbld3'
> | | | found build user `nixbld4'
> | | | found build user `nixbld5'
> | | | found build user `nixbld6'
> | | | found build user `nixbld7'
> | | | found build user `nixbld8'
> | | | found build user `nixbld9'
> | | | found build user `nixbld10'
> | | | trying user `nixbld1'
Yes, I see the same messages, then
| | | trying user `nixbld1'
| | | killing all processes running under uid `30001'
| | | setting up chroot environment in
`/nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-import.drv.chroot'
| | | executing builder `/nix/store/7ys18z66aih9dwzlwkj2jynk05a3xsds-
guile-bootstrap-2.0/bin/guile'
| | | @ build-started /nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-
module-import.drv /nix/store/fdzwc7ss7zbpg60n5h2kpp98l8sbjx9f-module-import
x86_64-linux /usr/local/guix-
git/var/log/nix/drvs/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-import.drv.bz2
| | building of `/nix/store/g1acby4isvypwrs141yl8wi9sr58wcr3-
gc-7.2alpha6.tar.gz.drv': all outputs substituted (maybe)
| | building of `/nix/store/2d4zra8i0j93qfvvbcxsd59fhphkgzfq-guile-
bootstrap-2.0.drv': created
| | building of `/nix/store/2d4zra8i0j93qfvvbcxsd59fhphkgzfq-guile-
bootstrap-2.0.drv': woken up
| | substitution of `/nix/store/davrb8rkw5q74y9j8b99jmq3ypm59pbc-
gc-7.2alpha6.tar.gz-guile-builder': created
| | substitution of `/nix/store/davrb8rkw5q74y9j8b99jmq3ypm59pbc-
gc-7.2alpha6.tar.gz-guile-builder': woken up
| | building of `/nix/store/2x28xlwzi66p0ydk6iwy008py9g7fg98-
perl-5.16.1.tar.gz.drv': all outputs substituted (maybe)
| | substitution of `/nix/store/aja3695yjwwm1zlwfqpfn4qi77s5mql1-
perl-5.16.1.tar.gz-guile-builder': created
| | substitution of `/nix/store/aja3695yjwwm1zlwfqpfn4qi77s5mql1-
perl-5.16.1.tar.gz-guile-builder': woken up
| | building of `/nix/store/imzn9c8knqvy65vxvm7b4w010xvx3qpi-module-
import.drv': trying to build
| | locking path `/nix/store/2psl4274zb67avwqhwa7d5afwjf8nk86-module-
import'
| | lock acquired on `/nix/store/2psl4274zb67avwqhwa7d5afwjf8nk86-
module-import.lock'
| | building path(s) `/nix/store/2psl4274zb67avwqhwa7d5afwjf8nk86-
module-import'
| | | found build user `nixbld1'
| | | found build user `nixbld2'
| | | found build user `nixbld3'
| | | found build user `nixbld4'
| | | found build user `nixbld5'
| | | found build user `nixbld6'
| | | found build user `nixbld7'
| | | found build user `nixbld8'
| | | found build user `nixbld9'
| | | found build user `nixbld10'
| | | trying user `nixbld1'
| | | trying user `nixbld2'
| | | trying user `nixbld3'
| | | trying user `nixbld4'
| | | trying user `nixbld5'
| | | trying user `nixbld6'
| | | trying user `nixbld7'
| | | trying user `nixbld8'
| | | trying user `nixbld9'
| | | trying user `nixbld10'
| building of `/nix/store/rqllv82x33mfqvma7zbj2kjdkxdqdwpx-
hello-2.8.drv': goal destroyed
(lots of goal destroyed)
| building of `/nix/store/hn55pz6ws2c26w8jbgmgzywmj9k6aghw-module-import-
compiled.drv': goal destroyed
| killing all processes running under uid `30001'
| recursively deleting path `/tmp/nix-build-module-import.drv-0'
| recursively deleting path `/nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-
module-import.drv.chroot'
| lock released on `/nix/store/fdzwc7ss7zbpg60n5h2kpp98l8sbjx9f-module-
import.lock'
| building of `/nix/store/6q7s15hqyqxkxnxxx1d7jj2rm6q28ib9-module-
import.drv': goal destroyed
error: build failed: all build users are currently in use; consider
creating additional users and adding them to the `nixbld' group
The full output is attached.
Andreas
[-- Attachment #1.2: Type: text/html, Size: 14792 bytes --]
[-- Attachment #2: output.bz2 --]
[-- Type: application/x-bzip, Size: 21651 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-15 20:25 ` Andreas Enge
@ 2012-12-15 22:59 ` Ludovic Courtès
2012-12-16 11:16 ` Andreas Enge
0 siblings, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-15 22:59 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Hi,
Andreas Enge <andreas@enge.fr> skribis:
> | | | found build user `nixbld1'
> | | | found build user `nixbld2'
> | | | found build user `nixbld3'
> | | | found build user `nixbld4'
> | | | found build user `nixbld5'
> | | | found build user `nixbld6'
> | | | found build user `nixbld7'
> | | | found build user `nixbld8'
> | | | found build user `nixbld9'
> | | | found build user `nixbld10'
> | | | trying user `nixbld1'
> | | | trying user `nixbld2'
> | | | trying user `nixbld3'
> | | | trying user `nixbld4'
> | | | trying user `nixbld5'
> | | | trying user `nixbld6'
> | | | trying user `nixbld7'
> | | | trying user `nixbld8'
> | | | trying user `nixbld9'
> | | | trying user `nixbld10'
First, guix-daemon is running as root, right?
That seems to suggest that there’s a lock on each of these. What does
‘ls -l /nix/var/nix/userpool/’ report?
Can you also try running guix-daemon in strace, and then check then
fcntl(fd, F_SETLKW, _) calls for user locks?
TIA,
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-15 22:59 ` Ludovic Courtès
@ 2012-12-16 11:16 ` Andreas Enge
2012-12-16 16:36 ` Ludovic Courtès
2012-12-17 21:16 ` Ludovic Courtès
0 siblings, 2 replies; 25+ messages in thread
From: Andreas Enge @ 2012-12-16 11:16 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1: Type: text/plain, Size: 900 bytes --]
Am Samstag, 15. Dezember 2012 schrieb Ludovic Courtès:
> First, guix-daemon is running as root, right?
Yes, in a separate terminal as
NIX_SUBSTITUTERS="" guix-daemon --build-users-group=nixbld
> That seems to suggest that there’s a lock on each of these. What does
> ‘ls -l /nix/var/nix/userpool/’ report?
-rw------- 1 root root 0 Dez 10 23:30 30001
/etc/group contains a line
nixbld:!:30000:nixbld1,nixbld2,nixbld3,nixbld4,nixbld5,nixbld6,nixbld7,nixbld8,nixbld9,nixbld10
and /etc/passwd:
nixbld1:x:30001:65534:Nix build user 1:/var/empty:/noshell
...
nixbld10:x:30001:65534:Nix build user 10:/var/empty:/noshell
> Can you also try running guix-daemon in strace, and then check then
> fcntl(fd, F_SETLKW, _) calls for user locks?
I find none of these:
strace ./pre-inst-env guix-build hello 2>&1 | grep fcntl | grep F_SETLKW
returns nothing.
Andreas
[-- Attachment #2: Type: text/html, Size: 5339 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-16 11:16 ` Andreas Enge
@ 2012-12-16 16:36 ` Ludovic Courtès
2012-12-16 16:56 ` Andreas Enge
2012-12-17 21:16 ` Ludovic Courtès
1 sibling, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-16 16:36 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Hi,
Andreas Enge <andreas@enge.fr> skribis:
> Am Samstag, 15. Dezember 2012 schrieb Ludovic Courtès:
[...]
>> Can you also try running guix-daemon in strace, and then check then
>> fcntl(fd, F_SETLKW, _) calls for user locks?
>
> I find none of these:
> strace ./pre-inst-env guix-build hello 2>&1 | grep fcntl | grep F_SETLKW
No, could you run guix-daemon in strace?
TIA,
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-16 16:36 ` Ludovic Courtès
@ 2012-12-16 16:56 ` Andreas Enge
2012-12-16 16:58 ` Ludovic Courtès
0 siblings, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-16 16:56 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1: Type: text/plain, Size: 2848 bytes --]
Am Sonntag, 16. Dezember 2012 schrieb Ludovic Courtès:
> No, could you run guix-daemon in strace?
Sorry, I did not read the message carefully enough. None of the
commands do appear.
Before executing "./pre-inst-env guix-build hello", the last lines of
output are
set_tid_address(0x7fa0b97469f0) = 7696
set_robust_list(0x7fa0b9746a00, 0x18) = 0
futex(0x7fffc31e130c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1,
NULL, 7fa0b9746720) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7fa0b80d7ad0, [], SA_RESTORER|SA_SIGINFO,
0x7fa0b80e1030}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fa0b80d7b60, [], SA_RESTORER|SA_RESTART|
SA_SIGINFO, 0x7fa0b80e1030}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
brk(0) = 0x14bb000
brk(0x14dc000) = 0x14dc000
futex(0x7fa0b8e149e4, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7fa0b8e149f0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
chdir("/") = 0
rt_sigaction(SIGCHLD, {0x406d70, ~[RTMIN RT_1], SA_RESTORER,
0x7fa0b83204f0}, {SIG_DFL, [], 0}, 8) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
lstat("/usr/local/guix-git/var/nix/daemon-socket", {st_mode=S_IFDIR|
S_ISGID|0755, st_size=4096, ...}) = 0
chdir("/usr/local/guix-git/var/nix/daemon-socket") = 0
unlink("/usr/local/guix-git/var/nix/daemon-socket/socket") = 0
umask(0111) = 022
bind(3, {sa_family=AF_FILE, path="./socket"}, 110) = 0
umask(022) = 0111
chdir("/") = 0
listen(3, 5) = 0
fcntl(3, F_GETFD) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
accept(3,
When starting the execution, only the following lines are printed:
{sa_family=AF_FILE, NULL}, [2]) = 4
fcntl(4, F_GETFD) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
getsockopt(4, SOL_SOCKET, SO_PEERCRED, "\24\36\0\0\350\3\0\0\350\3\0\0",
[12]) = 0
write(2, "accepted connection from pid 770"..., 44accepted connection from
pid 7700, uid 1000
) = 44
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7fa0b97469f0) = 7701
close(4) = 0
accept(3, spurious SIGPOLL
spurious SIGPOLL
spurious SIGPOLL
spurious SIGPOLL
spurious SIGPOLL
2187 operations
0x7fffc31dff10, [110]) = ? ERESTARTSYS (To be restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
wait4(-1, NULL, WNOHANG, NULL) = 7701
wait4(-1, NULL, WNOHANG, NULL) = -1 ECHILD (No child processes)
rt_sigreturn(0xffffffffffffffff) = -1 EINTR (Interrupted system
call)
accept(3,
Andreas
[-- Attachment #2: Type: text/html, Size: 11504 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-16 16:56 ` Andreas Enge
@ 2012-12-16 16:58 ` Ludovic Courtès
2012-12-16 17:33 ` Andreas Enge
0 siblings, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-16 16:58 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Andreas Enge <andreas@enge.fr> skribis:
> When starting the execution, only the following lines are printed:
> {sa_family=AF_FILE, NULL}, [2]) = 4
> fcntl(4, F_GETFD) = 0
> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> getsockopt(4, SOL_SOCKET, SO_PEERCRED, "\24\36\0\0\350\3\0\0\350\3\0\0",
> [12]) = 0
> write(2, "accepted connection from pid 770"..., 44accepted connection from
> pid 7700, uid 1000
> ) = 44
> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
Oh, actually since the daemon forks (last line above), you need -f:
# strace -o log -f guix-daemon --build-users-group=nixbld
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-16 16:58 ` Ludovic Courtès
@ 2012-12-16 17:33 ` Andreas Enge
0 siblings, 0 replies; 25+ messages in thread
From: Andreas Enge @ 2012-12-16 17:33 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1.1: Type: text/plain, Size: 306 bytes --]
Am Sonntag, 16. Dezember 2012 schrieb Ludovic Courtès:
> Oh, actually since the daemon forks (last line above), you need -f:
> # strace -o log -f guix-daemon --build-users-group=nixbld
Okay, output attached.
Towards the end, there are lots of "unfinished" and "bad file descriptor".
Andreas
[-- Attachment #1.2: Type: text/html, Size: 2131 bytes --]
[-- Attachment #2: log.bz2 --]
[-- Type: application/x-bzip, Size: 64701 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-16 11:16 ` Andreas Enge
2012-12-16 16:36 ` Ludovic Courtès
@ 2012-12-17 21:16 ` Ludovic Courtès
2012-12-17 21:48 ` Andreas Enge
1 sibling, 1 reply; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-17 21:16 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Hi Andreas,
Andreas Enge <andreas@enge.fr> skribis:
> and /etc/passwd:
> nixbld1:x:30001:65534:Nix build user 1:/var/empty:/noshell
> ...
> nixbld10:x:30001:65534:Nix build user 10:/var/empty:/noshell
It just occurred to me: here nixbld1 and nixbld10 have the same UID,
which is wrong. Can you check whether the others have a different UID?
Here I have:
--8<---------------cut here---------------start------------->8---
nixbld1:x:30001:30000:Nix build user 1:/var/empty:/run/current-system/sw/sbin/nologin
nixbld2:x:30002:30000:Nix build user 2:/var/empty:/run/current-system/sw/sbin/nologin
nixbld3:x:30003:30000:Nix build user 3:/var/empty:/run/current-system/sw/sbin/nologin
nixbld4:x:30004:30000:Nix build user 4:/var/empty:/run/current-system/sw/sbin/nologin
nixbld5:x:30005:30000:Nix build user 5:/var/empty:/run/current-system/sw/sbin/nologin
nixbld6:x:30006:30000:Nix build user 6:/var/empty:/run/current-system/sw/sbin/nologin
nixbld7:x:30007:30000:Nix build user 7:/var/empty:/run/current-system/sw/sbin/nologin
nixbld8:x:30008:30000:Nix build user 8:/var/empty:/run/current-system/sw/sbin/nologin
nixbld9:x:30009:30000:Nix build user 9:/var/empty:/run/current-system/sw/sbin/nologin
nixbld10:x:30010:30000:Nix build user 10:/var/empty:/run/current-system/sw/sbin/nologin
--8<---------------cut here---------------end--------------->8---
You should fix it with something like:
for b in `seq 1 10`; do usermod -u `expr 30000 + $b` nixbld$b ; done
Crossing fingers...
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-17 21:16 ` Ludovic Courtès
@ 2012-12-17 21:48 ` Andreas Enge
2012-12-17 22:25 ` Ludovic Courtès
0 siblings, 1 reply; 25+ messages in thread
From: Andreas Enge @ 2012-12-17 21:48 UTC (permalink / raw
To: Ludovic Courtès; +Cc: bug-guix
[-- Attachment #1: Type: text/plain, Size: 511 bytes --]
Am Montag, 17. Dezember 2012 schrieb Ludovic Courtès:
> It just occurred to me: here nixbld1 and nixbld10 have the same UID,
> which is wrong. Can you check whether the others have a different UID?
Silly me! I think that was it, a simple copy-and-paste error. Sorry for
taking your time. Strangely enough, the same setting worked for the
nix daemon.
Did anything in the base system change? I find myself redownloading gcc
and so on. Or does the switch to guix-daemon trigger a rebuild?
Andreas
[-- Attachment #2: Type: text/html, Size: 2739 bytes --]
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-17 21:48 ` Andreas Enge
@ 2012-12-17 22:25 ` Ludovic Courtès
0 siblings, 0 replies; 25+ messages in thread
From: Ludovic Courtès @ 2012-12-17 22:25 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Andreas Enge <andreas@enge.fr> skribis:
> Am Montag, 17. Dezember 2012 schrieb Ludovic Courtès:
>> It just occurred to me: here nixbld1 and nixbld10 have the same UID,
>> which is wrong. Can you check whether the others have a different UID?
>
> Silly me! I think that was it, a simple copy-and-paste error. Sorry for
> taking your time.
Good to know. :-)
> Strangely enough, the same setting worked for the nix daemon.
Hmm, interesting. We’d have to follow the logic in libstore/build.cc...
> Did anything in the base system change? I find myself redownloading gcc
> and so on. Or does the switch to guix-daemon trigger a rebuild?
Normally core packages didn’t change on ‘master’.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Integration of the Nix daemon
2012-12-13 23:10 ` Andreas Enge
@ 2013-01-15 23:03 ` Ludovic Courtès
0 siblings, 0 replies; 25+ messages in thread
From: Ludovic Courtès @ 2013-01-15 23:03 UTC (permalink / raw
To: Andreas Enge; +Cc: bug-guix
Andreas Enge <andreas@enge.fr> skribis:
> As well as
> guix-package --help
> One more reason to not have guix-package connect to the daemon when the
> options --help or --version are passed.
This is finally fixed, in commit c4d6453.
Thanks!
Ludo’.
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2013-01-15 23:03 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-12-03 22:13 Integration of the Nix daemon Ludovic Courtès
2012-12-13 21:17 ` Ludovic Courtès
2012-12-13 22:06 ` Andreas Enge
2012-12-13 22:56 ` Ludovic Courtès
2012-12-13 23:10 ` Andreas Enge
2013-01-15 23:03 ` Ludovic Courtès
2012-12-13 23:17 ` Andreas Enge
2012-12-14 10:07 ` Andreas Enge
2012-12-14 23:07 ` Ludovic Courtès
2012-12-14 23:23 ` Andreas Enge
2012-12-14 23:29 ` Andreas Enge
2012-12-15 14:48 ` Ludovic Courtès
2012-12-15 19:38 ` Andreas Enge
2012-12-15 14:26 ` Ludovic Courtès
2012-12-15 20:25 ` Andreas Enge
2012-12-15 22:59 ` Ludovic Courtès
2012-12-16 11:16 ` Andreas Enge
2012-12-16 16:36 ` Ludovic Courtès
2012-12-16 16:56 ` Andreas Enge
2012-12-16 16:58 ` Ludovic Courtès
2012-12-16 17:33 ` Andreas Enge
2012-12-17 21:16 ` Ludovic Courtès
2012-12-17 21:48 ` Andreas Enge
2012-12-17 22:25 ` Ludovic Courtès
2012-12-14 10:12 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).