bug#73601: failed to compute the derivation for Guix

[Fabrice Tudoret <fabrice.tudoret@univ-rennes1.fr>]

[-- Attachment #1.1: Type: text/plain, Size: 16497 bytes --]

Hi Simon,

I'm still struggling with the issue, but I found a kind of work around.

When I create the user home dir manually, the "guix pull" work's fine. 
So the trouble could come from a disorder with the automatic home dir 
creation.

I wish it's the right track. I keep digging.

Regards,

Fabrice TUDORET

Laboratoire du Traitement du Signal et de l'Image
INSERM U-1099. Université de Rennes 1
Campus de Beaulieu. Bât 22. 35042.  Rennes.  France

Le 14/10/2024 à 14:03, Fabrice Tudoret a écrit :
>
> Hi Simon,
>
> Thanks again for your involvement.
>
> I did my best to fullfill the tests you suggest. I put the output in 
> your text.
>
> Essentially the output are the same for the root and the users, so the 
> situation does not seem clearer to me, but I’m sure it will be 
> different for you.
>
> Regards
>
> Fabrice T
> Le 11/10/2024 à 18:30, Simon Tournier a écrit :
>> Hi Fabrice,
>>
>> On Fri, 11 Oct 2024 at 15:13, Fabrice Tudoret<fabrice.tudoret@univ-rennes1.fr> wrote:
>>
>>> GUIX seem's to work fine with the root account and the local users but
>>> not with ldap users.
>> Ah, that doesn’t ring a bell but maybe it’s related.  I don’t know.
>>
>>
>>> 1-
>> Just to be sure and since you have reinstalled, what is your Guix
>> revision?
> [root@cluster24 ~]# guix --version
> guix (GNU Guix) 7888351b9edd7b0199a973c75bc1c35897d9d7ef
> Copyright (C) 2024 the Guix authors
> License GPLv3+: GNU GPL version 3 or later<http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
>
>
>> On a side note, since it seems fine for the root account and here you
>> run it as root, I guess all is fine. :-)  Although I don’t have the
>> same… 🤔
>>
>>> [root@cluster24 ~]# cat /gnu/store/jc3vgcsplqsim3na80b0n2iilna5j6gx-Python-3.5.9.tar.xz.drv  | sed 's/)/)\n/g'
>>> Derive([("out","/gnu/store/cv4h89n30myf3nhjqnnahlbij2gaw21z-Python-3.5.9.tar.xz","","")
>>> ],[("/gnu/store/21c7pjahkh20mmzq2ivki57zwwvp6nwn-bootstrap-binaries-0.drv",["out"])
>>> ,("/gnu/store/5gf7f8awndhnf2gn2mzbfbqr3ix9aj80-module-import-compiled.drv",["out"])
>>> ,("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])
>>> ,("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])
>>> ],["/gnu/store/1s8jdafkyhz0p81l0j37yih9gbrb5gix-module-import","/gnu/store/h58cvdcdak4d87lw0fkvmkhan95ssljx-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/1s8jdafkyhz0p81l0j37yih9gbrb5gix-module-import","-C","/gnu/store/gz5rcilhcsc5amgxcgyxvn0s5px8sg80-module-import-compiled","/gnu/store/h58cvdcdak4d87lw0fkvmkhan95ssljx-Python-3.5.9.tar.xz-builder"],[("guix properties","((type . origin)
>>>   (patches . 0)
>>> )
>>> ")
>>> ,("out","/gnu/store/cv4h89n30myf3nhjqnnahlbij2gaw21z-Python-3.5.9.tar.xz")
>>> ])
>>>
>>>
>>> 2 -The script builder
>>> /gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder
>>> is well present in /gnu/store.
>> Ouf. :-)
>>
>>
>> Well, since it works as expected when run as root (#) and it fails when
>> run as a regular ($), I propose to diff various files in order to spot
>> what could be wrong.
>>
>> Some details about some internals – well my understanding and I’m
>> perhaps missing important points –, then maybe they will explain the
>> logic behind the exploration. :-)
>>
>> Roughly speaking, the items in the store look like:
>>
>>      /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-foobar-1.2.3
>>
>> where ’foobar-1.2.3’ is a “label” corresponding to the package and
>> ’xxx…’ is some hash.  This hash is the core of the content-addressed;
>> the one that allow the substitution, i.e., download the artefacts.
>>
>> Basically, this hash is computed by hashing the inputs and the script
>> builders.  Therefore somehow it builds a chain and the roots are named
>> fixed-outputs.  Fixed-outputs are items for which we known beforehand
>> the resulting hash.  Else we cannot know the hash beforehand because
>> it’s hard to know beforehand the checksum of the artefact since the
>> artefact is the result of the build process (compilation, etc.).
>>
>> In other words, the expectation is: the same inputs and the same builder
>> script returns the same store item.  And the derivation captures that.
>> Well, for sure the content of the store item on two machines is the same
>> only if the process is fully deterministic; another story. ;-)
>>
>> All that to say: if we scrutinize the derivations and the builder
>> scripts, then we will spot what introduces a difference.
>>
>> Aside, please note that two different derivations might produce the same
>> store item, see for example [1].
>>
>> Let’s go! :-)
>>
>>
>> a) Both root and regular must use the exact same Guix revision.
>>
>>    # As root
>>    # readlink -f (type -P guix)
>>    # guix describe
>>
>     [root@cluster24 ~]# readlink -f guix
>
>     /root/guix
>
>     [root@cluster24 ~]# type -P guix
>
>     /root/.config/guix/current/bin/guix
>
>     [root@cluster24 ~]# guix describe
>
>     Generation 3 Oct 14 2024 08:46:12 (current)
>
>     guix 7888351
>
>     repository URL:https://git.savannah.gnu.org/git/guix.git
>
>     branch: master
>
>     commit: 7888351b9edd7b0199a973c75bc1c35897d9d7ef
>
>>    $ As regular user
>>    $ readlink -f (type -P guix)
>>    $ guix describe
>
>     [fmenna@cluster24 ~]$ readlink -f guix
>
>     /NAS/home/fmenna/guix
>
>     [fmenna@cluster24 ~]$ type -P guix
>
>     /usr/local/bin/guix
>
>     [fmenna@cluster24 ~]$ guix describe
>
>     guix 7888351
>
>     repository URL:https://git.savannah.gnu.org/git/guix.git
>
>     branch: master
>
>     commit: 7888351b9edd7b0199a973c75bc1c35897d9d7ef
>
>> b) If that’s not the case, it isn’t an issue for the following but it’s
>>     better to know. :-)
>
>      It's seem ok.
>
>> c) Display the both derivations, as root and as regular user
>>
>>    # guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
>
>     [root@cluster24 ~]# guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
>     /gnu/store/mbnrrqjizy3dvma9i3l8g5yvzi2p04ys-Python-3.5.9.tar.xz.drv
>
>>    $ guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
>
>     [fmenna@cluster24 ~]$  guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
>     user with UID 255759 not found
>     /gnu/store/mbnrrqjizy3dvma9i3l8g5yvzi2p04ys-Python-3.5.9.tar.xz.drv
>
> I hope the message "user with UID 255759 not found" has nothing to do 
> with the issue.
>
>
>> The expected situation is: as root,
>>
>>    # guix build -e '(@@ (gnu packages commencement) python-boot0)' -S
>>
>> it just downloads stuff but as regular user:
>>
>>    $ guix build -e '(@@ (gnu packages commencement) python-boot0)' -S
>>
>> then it builds stuff and one build fails.
>>
>> Correct?
>
> There is no indication that it builds or fails.
>
>
>> d) Display the content of the both derivations, as root and and regular
>>     user.
>>
>>    drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
>>    cat $drv | sed 's/),/\n),/g'
>
>
> [root@cluster24 ~]# drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
>    cat $drv | sed 's/),/\n),/g'
> Derive([("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz","","")],[("/gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv",["out"]
> ),("/gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv",["out"]
> ),("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"]
> ),("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])],["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","-C","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],[("guix properties","((type . origin) (patches . 0))"
> ),("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz")])
> [fmenna@cluster24 ~]$  drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
>    cat $drv | sed 's/),/\n),/g'
> user with UID 255759 not found
> Derive([("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz","","")],[("/gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv",["out"]
> ),("/gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv",["out"]
> ),("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"]
> ),("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])],["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","-C","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],[("guix properties","((type . origin) (patches . 0))"
> ),("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz")])
>
>
> There is no difference in those output between root and regular user.
>
>
>> Now we can try to find what can be wrong. :-)
>>
>>
>> Roughly, the inputs are one fixed-output and these three others:
>>
>>      /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv
>>      /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv
>>      /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
>>
>> Well, please note that the hash of these derivations might vary but not
>> their output.  For instance,
>>
>>      # sed 's/]/\n]/g' \
>>         /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv \
>>        | head -1
>>      Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")
>>
>> Here the hash ’3895…’ might be different but not ’az8fi8…’.  The same
>> derivation produces the same store item though.
>>
>> Ok, let consider each.  I know that piping with “sed” is not handy and
>> an extension [2] ]is still pending… Anyway, please pipe the derivations
>> in order to extract the relevant information.
> [root@cluster24 ~]# sed 's/]/\n]/g' \
>        /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv | head -1
> Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")
> [fmenna@cluster24 ~]$ sed 's/]/\n]/g' \
> >  /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv | head -1
> Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")
>
>> 1. /gnu/store/…-guile-bootstrap: it only depends on fixed-outputs so the
>> only source of variation comes from the script builder: in my case,
>>
>>      /gnu/store/nmxl6qyj16bv4rx4irhg23r66gn752kd-build-bootstrap-guile.sh
>
> I have the same:
>
> [root@cluster24 ~]# find /gnu/store -name "*build-bootstrap-guile.sh"
> /gnu/store/nmxl6qyj16bv4rx4irhg23r66gn752kd-build-bootstrap-guile.sh
>
>> I expect you have the same.  As root, you have because it’s the same
>> derivation, namely
>> /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv.
>>
>> What about the regular user?
>
> The same too.
>
>> 2. /gnu/store/…-bootstrap-binaries: the inputs reads,
>>
>>      /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
>>      /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv
>>      /gnu/store/nf9h1l2apgjq1gjdv1ynpgqrmaj1bawb-static-binaries.tar.xz.drv
>>      /gnu/store/sik0rp63q7hz4wry27v3ynljdmqvg194-xz.drv
>>      /gnu/store/zn60yim3km5qx8d0vzvf9izfxzfdy4x0-tar.drv
>>
>> Assuming #1 (identical guile-bootstrap) and the only non fixed-output is
>> module-import-compiled.  Or the script builder.
>>
>> Do you have the same script builder, namely
>>
>>    /gnu/store/kv2vjnp7f75wnq8rp579ihz96knqp0sl-bootstrap-binaries-0-builder
>>
>> ?  I guess you have the same.  Therefore, I guess the issue is from
>> module-import-compiled.
>
> I've the same:
>
> [root@cluster24 ~]# find /gnu/store -name "*bootstrap-binaries-0-builder"
> /gnu/store/kv2vjnp7f75wnq8rp579ihz96knqp0sl-bootstrap-binaries-0-builder
>
>
>> 3. /gnu/store/…-module-import-compiled: On my machine, it reads:
>>
>> --8<---------------cut here---------------start------------->8---
>> Derive
>> ([("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","","")]
>>   ,[("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])]
>>   ,["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/l36kgiw25r2a4vllwad18jg8jnkvmzda-build-utils.scm","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
>>   ,"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
>>   ,[("GUILE_WARN_DEPRECATED","no")
>>     ,("extensions","")
>>     ,("module count","1")
>>     ,("modules","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import")
>>     ,("optimization level","1")
>>     ,("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled")
>>     ,("preferLocalBuild","1")])
>> --8<---------------cut here---------------end--------------->8---
>>
>> I guess you also have the same build-utils.scm.  Don’t you?
>
> I've the exactly the same content in the file 
> /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
>
> [root@cluster24 ~]# cat /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
> Derive([("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","","")]
> ,[("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])]
> ,["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/l36kgiw25r2a4vllwad18jg8jnkvmzda-build-utils.scm","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
> ,"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
> ,[("GUILE_WARN_DEPRECATED","no")
> ,("extensions","")
> ,("module count","1")
> ,("modules","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import")
> ,("optimization level","1"),("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled")
> ,("preferLocalBuild","1")])
>
> About the build-utils.scm, they are also identical:
>
> /root/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/tests/build-utils.scm
> /NAS/home/fmenna/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/tests/build-utils.scm
>
>> Therefore, I guess the difference is either an environment variable (the
>> list starting with GUILE_WARN_DEPRECATED) or the file
>> /gnu/store/…-qcompile-modules.  Hum, it would be weird that it comes
>> from compile-modules…
>
>
>
>> Ah my train is at destination.  To be continued… Let me know!
>>
>> Cheers,
>> simon
>>
>>
>> 1: Re: Derivations differ between computers?
>> zimoun<zimon.toutoune@gmail.com>
>> Fri, 26 Nov 2021 01:49:00 +0100
>> id:86lf1bspvn.fsf@gmail.com
>> https://lists.gnu.org/archive/html/guix-devel/2021-11
>> https://yhetil.org/guix/86lf1bspvn.fsf@gmail.cmo
>>
>> 2: Guix extension to display derivation (and rewrite fixed-output)
>> Simon Tournier<zimon.toutoune@gmail.com>
>> Fri, 12 Apr 2024 20:28:11 +0200
>> id:87ttk6phac.fsf@gmail.com
>> https://lists.gnu.org/archive/html/guix-devel/2024-04
>> https://yhetil.org/guix/87ttk6phac.fsf@gmail.com

[-- Attachment #1.2: Type: text/html, Size: 22232 bytes --]

[-- Attachment #2: Signature cryptographique S/MIME --]
[-- Type: application/pkcs7-signature, Size: 6087 bytes --]