bug#33285: Installing, then removing, a package yields a different profile

bug#33285: Installing, then removing, a package yields a different profile

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1678 bytes --]

Hi,

I've noticed that if I do the following...

guix package -p .guix-profile -i hello
guix package -p .guix-profile -i coreutils
guix package -p .guix-profile -r coreutils

...I get profile generations like the following:

.guix-profile-1-link  ->  /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile
.guix-profile-2-link  ->  /gnu/store/m6cwnnxcccnd8bga52jxn74vm67n7ry1-profile
.guix-profile-3-link  ->  /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile

However, the 1st and 3rd generations are practically the same:

--8<---------------cut here---------------start------------->8---
$ diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile
diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile/etc/profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile/etc/profile
11c11
< export PATH="${GUIX_PROFILE:-/gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile}/bin${PATH:+:}$PATH"
---
> export PATH="${GUIX_PROFILE:-/gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile}/bin${PATH:+:}$PATH"
[1] marusich@garuda.local:/tmp/tmp.ZOTdkBr4rD
$ 
--8<---------------cut here---------------end--------------->8---

Now, this isn't really a problem, since it isn't wasting space, and it
doesn't take long to build.  It's more of a curiosity.  Why is it that
profile generations 1 and 3 don't both point to the same profile in the
store?  Is this expected behavior?

I wasn't sure what would happen when I tried this.  I had guessed
(incorrectly) that generation 3 would point to the same profile that had
been built for generation 1, so I was a little surprised by this.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#33285: Installing, then removing, a package yields a different profile

[Ludovic Courtès]

Hi,

Chris Marusich <cmmarusich@gmail.com> skribis:

> I've noticed that if I do the following...
>
> guix package -p .guix-profile -i hello
> guix package -p .guix-profile -i coreutils
> guix package -p .guix-profile -r coreutils
>
> ...I get profile generations like the following:
>
> .guix-profile-1-link  ->  /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile
> .guix-profile-2-link  ->  /gnu/store/m6cwnnxcccnd8bga52jxn74vm67n7ry1-profile
> .guix-profile-3-link  ->  /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile
>
> However, the 1st and 3rd generations are practically the same:
>
> $ diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile
> diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile/etc/profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile/etc/profile
> 11c11
> < export PATH="${GUIX_PROFILE:-/gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile}/bin${PATH:+:}$PATH"
> ---
>> export PATH="${GUIX_PROFILE:-/gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile}/bin${PATH:+:}$PATH"
> [1] marusich@garuda.local:/tmp/tmp.ZOTdkBr4rD
> $ 
>
> Now, this isn't really a problem, since it isn't wasting space, and it
> doesn't take long to build.  It's more of a curiosity.  Why is it that
> profile generations 1 and 3 don't both point to the same profile in the
> store?  Is this expected behavior?

It’s not!  Could you look at the derivers of these two profiles (use
‘guix gc --derivers’) and see what differs?

Thanks,
Ludo’.

bug#33285: Installing, then removing, a package yields a different profile

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1878 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Chris Marusich <cmmarusich@gmail.com> skribis:
>
>> [...]
>>
>> Why is it that profile generations 1 and 3 don't both point to the
>> same profile in the store?  Is this expected behavior?
>
> It’s not!  Could you look at the derivers of these two profiles (use
> ‘guix gc --derivers’) and see what differs?

Due to preparations for SeaGL and other matters, I'm afraid I won't have
much extra time until the end of this month.  However, I've peeked at
the derivers, and I may have a hint for now.

The derivers' builders differ as follows:

--8<---------------cut here---------------start------------->8---
$ diff -u  <(cat /gnu/store/nxjngfdrnbid45vibaa3rwyahs8mxdy8-profile-builder | tr ' ' '\n') <(cat /gnu/store/ibgc1gmwmqpvq3shls19b6w7x96r773m-profile-builder | tr ' ' '\n')
--- /dev/fd/63	2018-11-06 22:57:06.113873035 -0800
+++ /dev/fd/62	2018-11-06 22:57:06.113873035 -0800
@@ -38,10 +38,10 @@
 getenv)
 "out")
 (quote
-("/gnu/store/cb6cxb0k18a648qclh7zd7pvpsh1imny-info-dir"
-"/gnu/store/s616mj87kgz41g89b46fgr12y85gldrv-manual-database"
-"/gnu/store/1mj3mlqc28p68q0lnplv217vv4v9fslc-fonts-dir"
-"/gnu/store/g2q9r93pylmfqx4sw3k5gxixg8q2ksb1-ca-certificate-bundle"
+("/gnu/store/vh0ngq20a6x7zshjpv4qnd39p6sfh892-info-dir"
+"/gnu/store/ha6an8rfxadrk3slg9rxwdirv3a64kbz-manual-database"
+"/gnu/store/6cn1mrlyp1xlvln6msx25h822zjk62c3-fonts-dir"
+"/gnu/store/r5vlx8c33xs9g041hi7fyjc0fy2yndrn-ca-certificate-bundle"
 "/gnu/store/bihfrh609gkxb9dp7n96wlpigiv3krfy-hello-2.10"))
 #:symlink
 symlink
--8<---------------cut here---------------end--------------->8---

Notably, they DO both seem to use the same store path for hello.  The
only differing paths look like they were produced by profile hooks.
Perhaps this issue has something to do with profile hooks?

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#33285: Installing, then removing, a package yields a different profile

[Ricardo Wurmus]

Hi,

Chris Marusich <cmmarusich@gmail.com> writes:

> The only differing paths look like they were produced by profile
> hooks.  Perhaps this issue has something to do with profile hooks?

This seems to be the case.  Here are the derivations for the info-dir
hook for the first and then the third profile:

--8<---------------cut here---------------start------------->8---
Derive
([("out","/gnu/store/vq1frsx6jkvk957mi2sx0rl629r31vi0-info-dir","","")]
 ,[("/gnu/store/32dcpfx9chyf1z3jhv41r7hx2a6vklw2-glibc-utf8-locales-2.28.drv",["out"])
   ,("/gnu/store/h0905d946s3xarcfakan8jnblcdqxs3a-hello-2.10.drv",["out"])
   ,("/gnu/store/jbnidakwvhr84rp8p8icdg1lnczjvi6w-gzip-1.9.drv",["out"])
   ,("/gnu/store/n9zlvq692s12aarbrfppz7ayzggxfkqx-guile-2.2.4.drv",["out"])
   ,("/gnu/store/p6ml6z1d3y387y06mbcvpnf19ik2qywh-module-import-compiled.drv",["out"])
   ,("/gnu/store/x62102cgam20b3g8vfdvw06snvznx12k-texinfo-6.5.drv",["out"])]
 ,["/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import"]
 ,"x86_64-linux","/gnu/store/r658y3cgpnf99nxjxqgjiaizx20ac4k0-guile-2.2.4/bin/guile",["--no-auto-compile","-L","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import","-C","/gnu/store/qkqypgkam1fncl32l5qklplqh3a6k3fz-module-import-compiled","/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder"]
 ,[("allowSubstitutes","0")
   ,("guix properties","((type . profile-hook) (hook . info-dir))")
   ,("out","/gnu/store/vq1frsx6jkvk957mi2sx0rl629r31vi0-info-dir")
   ,("preferLocalBuild","1")])
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
Derive
([("out","/gnu/store/2ivmnbir288p7w7xyj1bb338h30q00ma-info-dir","","")]
 ,[("/gnu/store/32dcpfx9chyf1z3jhv41r7hx2a6vklw2-glibc-utf8-locales-2.28.drv",["out"])
   ,("/gnu/store/jbnidakwvhr84rp8p8icdg1lnczjvi6w-gzip-1.9.drv",["out"])
   ,("/gnu/store/n9zlvq692s12aarbrfppz7ayzggxfkqx-guile-2.2.4.drv",["out"])
   ,("/gnu/store/p6ml6z1d3y387y06mbcvpnf19ik2qywh-module-import-compiled.drv",["out"])
   ,("/gnu/store/x62102cgam20b3g8vfdvw06snvznx12k-texinfo-6.5.drv",["out"])]
 ,["/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10","/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import"]
 ,"x86_64-linux","/gnu/store/r658y3cgpnf99nxjxqgjiaizx20ac4k0-guile-2.2.4/bin/guile",["--no-auto-compile","-L","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import","-C","/gnu/store/qkqypgkam1fncl32l5qklplqh3a6k3fz-module-import-compiled","/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder"]
 ,[("allowSubstitutes","0")
   ,("guix properties","((type . profile-hook) (hook . info-dir))")
   ,("out","/gnu/store/2ivmnbir288p7w7xyj1bb338h30q00ma-info-dir")
   ,("preferLocalBuild","1")])
--8<---------------cut here---------------end--------------->8---

Note that they differ in the inclusion of /gnu/store/h0905d946s3xarcfakan8jnblcdqxs3a-hello-2.10.drv as
a derivation input.

I guess that the problem might be in “profile-derivation”, which defines
“inputs” like this:

    (define inputs
      (append (filter-map (lambda (drv)
                            (and (derivation? drv)
                                 (gexp-input drv)))
                          extras)
              (manifest-inputs manifest)))

I pk’d at the value returned by manifest-inputs and found this:

--8<---------------cut here---------------start------------->8---
$ guix package -p .guix-profile -i hello
The following package will be installed:
   hello	2.10	/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10


;;; ((#<gexp-input #<package hello@2.10 gnu/packages/base.scm:68 31e4d80>:out>))
1 package in profile
The following environment variable definitions may be needed:
   export PATH=".guix-profile/bin${PATH:+:}$PATH"

$ guix package -p .guix-profile -i coreutils
The following package will be installed:
   coreutils	8.30	/gnu/store/xzgfh8kdxilyymi1a8i97b1nci1iybhi-coreutils-8.30


;;; ((#<gexp-input #<package coreutils@8.30 gnu/packages/base.scm:338 39a7000>:out> #<gexp-input "/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10":out>))
2 packages in profile
The following environment variable definitions may be needed:
   export PATH=".guix-profile/bin${PATH:+:}$PATH"


$ guix package -p .guix-profile -r coreutils
The following package will be removed:
   coreutils	8.30	/gnu/store/xzgfh8kdxilyymi1a8i97b1nci1iybhi-coreutils-8.30


;;; ((#<gexp-input "/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10":out>))
1 package in profile
The following environment variable definitions may be needed:
   export PATH=".guix-profile/bin${PATH:+:}$PATH"
--8<---------------cut here---------------end--------------->8---

Should manifest-inputs distinguish between a package entry and a file
entry and paper over the differences?

--
Ricardo

bug#33285: Installing, then removing, a package yields a different profile

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Note that they differ in the inclusion of /gnu/store/h0905d946s3xarcfakan8jnblcdqxs3a-hello-2.10.drv as
> a derivation input.
>
> I guess that the problem might be in “profile-derivation”, which defines
> “inputs” like this:
>
>     (define inputs
>       (append (filter-map (lambda (drv)
>                             (and (derivation? drv)
>                                  (gexp-input drv)))
>                           extras)
>               (manifest-inputs manifest)))
>
> I pk’d at the value returned by manifest-inputs and found this:
>
> $ guix package -p .guix-profile -i hello
> The following package will be installed:
>    hello	2.10	/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10
>
>
> ;;; ((#<gexp-input #<package hello@2.10 gnu/packages/base.scm:68 31e4d80>:out>))
> 1 package in profile
> The following environment variable definitions may be needed:
>    export PATH=".guix-profile/bin${PATH:+:}$PATH"
>
> $ guix package -p .guix-profile -i coreutils
> The following package will be installed:
>    coreutils	8.30	/gnu/store/xzgfh8kdxilyymi1a8i97b1nci1iybhi-coreutils-8.30
>
>
> ;;; ((#<gexp-input #<package coreutils@8.30 gnu/packages/base.scm:338 39a7000>:out> #<gexp-input "/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10":out>))

Good catch!

> Should manifest-inputs distinguish between a package entry and a file
> entry and paper over the differences?

Actually, to do things correctly, we should really store the .drv in the
‘manifest’ file.  That way, manifest entries would always contains a
valid “lowerable” object (a package or a derivation record), as opposed
to a store reference that happens to be valid.

I’ll see if I can come up with a patch for this.

Thanks,
Ludo’.

bug#33285: Installing, then removing, a package yields a different profile

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> Actually, to do things correctly, we should really store the .drv in the
> ‘manifest’ file.  That way, manifest entries would always contains a
> valid “lowerable” object (a package or a derivation record), as opposed
> to a store reference that happens to be valid.

On second thought that’s probably not a great idea: we’d retain
references to the .drv themselves, which can end up taking a bit of
space, especially if --gc-keep-outputs is in use.

So… the more I think about it the more I’m tempted to punt on this one.

The discrepancy we see here does not happen when using --manifest, which
is probably what matters most.

Thoughts?

Ludo’.

bug#33285: Installing, then removing, a package yields a different profile

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Actually, to do things correctly, we should really store the .drv in the
>> ‘manifest’ file.  That way, manifest entries would always contains a
>> valid “lowerable” object (a package or a derivation record), as opposed
>> to a store reference that happens to be valid.
>
> On second thought that’s probably not a great idea: we’d retain
> references to the .drv themselves, which can end up taking a bit of
> space, especially if --gc-keep-outputs is in use.
>
> So… the more I think about it the more I’m tempted to punt on this one.
>
> The discrepancy we see here does not happen when using --manifest, which
> is probably what matters most.

I think it’s fine to leave it as it is, then.  “--install” and
“--remove” are stateful and should be expected to have quirks like this.
(E.g. upgrading Guix in between two “--install”s can lead to
a mosaic of a profile that could not be produced any other way.)

“--manifest” is the recommended way to get reproducible profile
generations, so if that’s working fine I’m okay with closing this as
wontfix.  Chris, what do you think?

Thanks for investigating!

--
Ricardo

bug#33285: Installing, then removing, a package yields a different profile

[Jelle Licht]

Ricardo Wurmus <rekado@elephly.net> writes:

> [snip]
>
> I think it’s fine to leave it as it is, then.  “--install” and
> “--remove” are stateful and should be expected to have quirks like this.
> (E.g. upgrading Guix in between two “--install”s can lead to
> a mosaic of a profile that could not be produced any other way.)

This seems like a very reasonable perspective on this issue, so I
concur. Should we state this explicitly somewhere in the manual?

bug#33285: Installing, then removing, a package yields a different profile

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1734 bytes --]

Jelle Licht <jlicht@fsfe.org> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> [snip]
>>
>> I think it’s fine to leave it as it is, then.  “--install” and
>> “--remove” are stateful and should be expected to have quirks like this.
>> (E.g. upgrading Guix in between two “--install”s can lead to
>> a mosaic of a profile that could not be produced any other way.)
>
> This seems like a very reasonable perspective on this issue, so I
> concur. Should we state this explicitly somewhere in the manual?

Thinking about it that way, I agree it makes sense as is.  I think it's
valid to say this isn't a bug, then.  I've confirmed that "adding" and
then "removing" a package by using a stateless method (e.g., the "guix
package -m" method) results in the expected behavior: the first and
third profile generations refer to exactly the same profile:

--8<---------------cut here---------------start------------->8---
$ ls -al /tmp/test-profile*
lrwxrwxrwx 1 marusich users 19 Feb  8 19:39 /tmp/test-profile -> test-profile-3-link
lrwxrwxrwx 1 marusich users 51 Feb  8 19:39 /tmp/test-profile-1-link -> /gnu/store/7j6gy2xvyz7zzpccwsssbxxzykv6q0zg-profile
lrwxrwxrwx 1 marusich users 51 Feb  8 19:39 /tmp/test-profile-2-link -> /gnu/store/yz9qskdl926mdy1i63yzsqwhcciswv90-profile
lrwxrwxrwx 1 marusich users 51 Feb  8 19:39 /tmp/test-profile-3-link -> /gnu/store/7j6gy2xvyz7zzpccwsssbxxzykv6q0zg-profile
--8<---------------cut here---------------end--------------->8---

I'm satisfied.  I'm honestly not sure it's worth mentioning in the
manual.  I feel like explaining it would add little value and just
clutter up the documentation.  This was mainly a curiosity for me.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#33285: Installing, then removing, a package yields a different profile

[Ricardo Wurmus]

Chris Marusich <cmmarusich@gmail.com> writes:

> Thinking about it that way, I agree it makes sense as is.  I think it's
> valid to say this isn't a bug, then.

Okay, I’m closing this bug.
Thanks for bringing this up!

-- 
Ricardo