* Two copies of same library in store after cleaning generations and gc @ 2022-08-14 11:31 Mike Jones 2022-08-18 12:13 ` Hartmut Goebel 0 siblings, 1 reply; 7+ messages in thread From: Mike Jones @ 2022-08-14 11:31 UTC (permalink / raw) To: help-guix Hi Guix! I've been running Guix System for a few weeks now, and I've noticed that I have two copies of some libraries in my store (e.g. mesa, gtk) even after updating all packages, deleting old profile generations, and running gc. This isn't a problem, as I have plenty of disk space, but I'm just curious to understand why this happens. For example, I have both /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 /gnu/store/wj1nipayc74yalfmdg7wwimkfc1macbl-mesa-21.3.8 Using 'guix gc --referrers', I can see that: - The first copy is referred to by all applications in my profile that link against mesa, e.g. icecat. - The second copy is referred to directly by my profile. Indeed, I can see that, for example, ~/.guix-profile/lib/libGL.so is a symlink to /gnu/store/wj1nipayc74yalfmdg7wwimkfc1macbl-mesa-21.3.8/lib/libGL.so. So my question boils down to: why is the copy of mesa in ~/.guix-profile/lib different to the one to which programs in the same profile are linked? And is there any guix command I can use to inspect exactly how/why these copies differ? Thanks! - Mike ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc 2022-08-14 11:31 Two copies of same library in store after cleaning generations and gc Mike Jones @ 2022-08-18 12:13 ` Hartmut Goebel 2022-08-18 20:30 ` Mike Jones 0 siblings, 1 reply; 7+ messages in thread From: Hartmut Goebel @ 2022-08-18 12:13 UTC (permalink / raw) To: Mike Jones, help-guix Am 14.08.22 um 13:31 schrieb Mike Jones: > So my question boils down to: why is the copy of mesa in > ~/.guix-profile/lib different to the one to which programs in the same > profile are linked? And is there any guix command I can use to inspect > exactly how/why these copies differ? I assume you installed mesa into your profile — or some other package installed in your profile has mesa as „propagated input“. Try „guix package -l“ to see whether it is installed in your profile. -- Regards Hartmut Goebel | Hartmut Goebel |h.goebel@crazy-compilers.com | |www.crazy-compilers.com | compilers which you thought are impossible | ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc 2022-08-18 12:13 ` Hartmut Goebel @ 2022-08-18 20:30 ` Mike Jones 2022-08-19 6:13 ` Ricardo Wurmus 0 siblings, 1 reply; 7+ messages in thread From: Mike Jones @ 2022-08-18 20:30 UTC (permalink / raw) To: Hartmut Goebel, guix.rc7ys--- via Hello, Thanks for the hints. > I assume you installed mesa into your profile — or some other package > installed in your profile has mesa as „propagated input“. > > Try „guix package -l“ to see whether it is installed in your profile. I haven't installed mesa explicitly, and it doesn't show up in `guix package -l`, so I think you must be right that it's coming from a propagated input. I still wonder, though, why there are two different builds of mesa appearing. I suspected it might be related to grafting, so I poked around in /gnu/store and found the .drv files that produced these two different copies. They both seem to be the result of a grafting process. If I rebuild my profile with --no-grafts, the mesa used by programs and the one in the profile are the same, so I'll just put this down to some weirdness with grafting. - Mike ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc 2022-08-18 20:30 ` Mike Jones @ 2022-08-19 6:13 ` Ricardo Wurmus 2022-08-19 9:06 ` zimoun 0 siblings, 1 reply; 7+ messages in thread From: Ricardo Wurmus @ 2022-08-19 6:13 UTC (permalink / raw) To: Mike Jones; +Cc: Hartmut Goebel, help-guix "Mike Jones" <mike@mjones.io> writes: > If I rebuild my profile with --no-grafts, the mesa used by programs and > the one in the profile are the same, so I'll just put this down to some > weirdness with grafting. This sounds like grafts working as intended. Do you also have more than one instance of icecat – one using the graft and one without? (Due to deduplication in /gnu/store this is not as wasteful as it may seem.) -- Ricardo ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc 2022-08-19 6:13 ` Ricardo Wurmus @ 2022-08-19 9:06 ` zimoun 2022-08-19 11:37 ` Mike Jones 0 siblings, 1 reply; 7+ messages in thread From: zimoun @ 2022-08-19 9:06 UTC (permalink / raw) To: Ricardo Wurmus, Mike Jones; +Cc: Hartmut Goebel, help-guix Hi, On ven., 19 août 2022 at 08:13, Ricardo Wurmus <rekado@elephly.net> wrote: > This sounds like grafts working as intended. Do you also have more than > one instance of icecat – one using the graft and one without? > > (Due to deduplication in /gnu/store this is not as wasteful as it may > seem.) Using Guix 8f0d45c, --8<---------------cut here---------------start------------->8--- $ guix build mesa /gnu/store/5y054ijr0dqpd6v4wlby9alhbzmha033-mesa-21.3.8-bin /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 $ guix build mesa --no-grafts /gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin /gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8 $ guix gc --references $(guix build icecat) | grep mesa /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 --8<---------------cut here---------------end--------------->8--- it can be confusing… Let examine. :-) First, let consider mesa referred by icecat. --8<---------------cut here---------------start------------->8--- $ guix gc --derivers /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 /gnu/store/qqa192m2cx4qrlvhr1dg7vrc49jqa84h-mesa-21.3.8.drv /gnu/store/y2yxw4zd4p916lby3sibmskdv860zvpa-mesa-21.3.8.drv --8<---------------cut here---------------end--------------->8--- It means that the both derivations, /gnu/store/qqa192m2cx4qrlvhr1dg7vrc49jqa84h-mesa-21.3.8.drv /gnu/store/y2yxw4zd4p916lby3sibmskdv860zvpa-mesa-21.3.8.drv have the same output: /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 Yes, that’s surprising but possible. From /gnu/store/qqa192m2cx4qrlvhr1dg7vrc49jqa84h-mesa-21.3.8.drv, the guile-builder reads: --8<---------------cut here---------------start------------->8--- (define %build-inputs (quote (("x" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8") [...] ((old-outputs (quote (("out" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8")))) --8<---------------cut here---------------end--------------->8--- which grafts, I guess. Note that this derivation lists the other derivation /gnu/store/0yd0gxbcrkh32zy4q8dkpyyv2xpvm1mh-mesa-21.3.8.drv. From /gnu/store/y2yxw4zd4p916lby3sibmskdv860zvpa-mesa-21.3.8.drv, the guile-builder is exactly the same (/gnu/store/a0h343…-mesa-21.3.8-guile-builder). However, the derivation lists /gnu/store/ylx0p9sl6qla4qsmp6dsvaqgd7g3m1rw-mesa-21.3.8.drv. These two derivations, /gnu/store/0yd0gxbcrkh32zy4q8dkpyyv2xpvm1mh-mesa-21.3.8.drv /gnu/store/ylx0p9sl6qla4qsmp6dsvaqgd7g3m1rw-mesa-21.3.8.drv. have the same outputs; which reads: --8<---------------cut here---------------start------------->8--- Derive ([("bin","/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin","","") ,("out","/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8","","")] --8<---------------cut here---------------end--------------->8--- other said, the ungrafted version. So far, so good! Second, let consider the grafted build, --8<---------------cut here---------------start------------->8--- $ guix gc --derivers /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 /gnu/store/ji6h6y5jizwhsx2jzw67myk5fhiv87va-mesa-21.3.8.drv --8<---------------cut here---------------end--------------->8--- and the guile-builder (/gnu/store/j4j58b…-mesa-21.3.8-guile-builder) contains, --8<---------------cut here---------------start------------->8--- (define %build-inputs (quote (("x" . "/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin") ("x" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8") [...] ((old-outputs (quote (("bin" . "/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin") ("out" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8")))) --8<---------------cut here---------------end--------------->8--- and that’s why the hash 74ags388lb… (from “guix build mesa”) is different from lcqz4q383… (from icecat). This derivation /gnu/store/ji6h6y5jizwhsx2jzw67myk5fhiv87va-mesa-21.3.8.drv lists the other derivation, /gnu/store/0yd0gxbcrkh32zy4q8dkpyyv2xpvm1mh-mesa-21.3.8.drv. Wait, it has already appeared below. :-) Well, all seems consistent, isn’t it? I hope now it is less confusing. However, something still appears to me confusing: --8<---------------cut here---------------start------------->8--- $ diff -r --no-dereference /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 Binary files /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8/lib/dri/i830_dri.so and /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8/lib/dri/i830_dri.so differ [...] diff -r --no-dereference /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8/lib/pkgconfig/dri.pc /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8/lib/pkgconfig/dri.pc 1c1 < prefix=/gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 --- > prefix=/gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 [...] --8<---------------cut here---------------end--------------->8--- Other said, the grafted item is duplicated. Right? Therefore, my question is: is deduplication optimal? Or better worded: what is the waste by the duplicated grafted items? Cheers, simon ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc 2022-08-19 9:06 ` zimoun @ 2022-08-19 11:37 ` Mike Jones 2022-08-22 15:49 ` zimoun 0 siblings, 1 reply; 7+ messages in thread From: Mike Jones @ 2022-08-19 11:37 UTC (permalink / raw) To: zimoun, Ricardo Wurmus; +Cc: Hartmut Goebel, guix.rc7ys--- via Hi all! On Fri, 19 Aug 2022, at 7:13 AM, Ricardo Wurmus wrote: > This sounds like grafts working as intended. Do you also have more > than one instance of icecat – one using the graft and one without? After building the profile do I indeed have two instances of icecat in the store. But when I run `guix gc`, the ungrafted one is of course deleted, and I'm left with a single instance. With mesa, however, there are always two active instances of mesa (also some other libraries, e.g. gtk) referenced by the profile. I expected that there would only be one (grafted) instance. On Fri, 19 Aug 2022, at 10:06 AM, zimoun wrote: > Well, all seems consistent, isn’t it? I hope now it is less > confusing. Yeah, thanks for working through this - I understand better now :) So we have one original ungrafted output, which is: /gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8 This then gets separately grafted twice to produce two new outputs: /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 (result of `guix build mesa`) /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 (referred to by icecat) But I wonder why icecat refers to a different mesa to `guix build mesa`. Running $ diffoscope /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 \ /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8` (takes quite a long time to run!) shows me that many of the files are different, but only in self-references. I also diffed the guile-builder scripts for the two grafted mesa instances. Here's the first difference: - (map (lambda (o) (cons o (getenv o))) - '("bin" "out"))) + (map (lambda (o) (cons o (getenv o))) '("out"))) It seems that one is a graft of just the default output, while the other also includes the "bin" output! I think this explains why there are two separate grafts, that produce the same (minus self-references) output. Presumably these aren't deduplicated due to the differing self-references? - Mike ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc 2022-08-19 11:37 ` Mike Jones @ 2022-08-22 15:49 ` zimoun 0 siblings, 0 replies; 7+ messages in thread From: zimoun @ 2022-08-22 15:49 UTC (permalink / raw) To: Mike Jones, Ricardo Wurmus; +Cc: Hartmut Goebel, guix.rc7ys--- via Hi, On ven., 19 août 2022 at 12:37, "Mike Jones" <mike@mjones.io> wrote: > Presumably these aren't deduplicated due to the differing > self-references? Yes. Note that this happens for only the packages with multi-outputs impacted by grafted ones. If I am correct, using Guix 65cabb0, the number of packages with multi-outputs is 681 (see below). On this number, a subset will be impacted by the ~11 grafted packages. Cheers, simon --8<---------------cut here---------------start------------->8--- scheme@(guix-user)> ,use(ice-9 match) scheme@(guix-user)> ,use(gnu) scheme@(guix-user)> ,use(guix) scheme@(guix-user)> (length (fold-packages (lambda (package result) (match (package-outputs package) ((x y z ...) (cons package result)) (_ result))) '())) $1 = 681 scheme@(guix-user)> (length (fold-packages (lambda (package result) (if (package-replacement package) (cons package result) result)) '())) $2 = 11 --8<---------------cut here---------------end--------------->8--- ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-08-22 18:44 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-08-14 11:31 Two copies of same library in store after cleaning generations and gc Mike Jones 2022-08-18 12:13 ` Hartmut Goebel 2022-08-18 20:30 ` Mike Jones 2022-08-19 6:13 ` Ricardo Wurmus 2022-08-19 9:06 ` zimoun 2022-08-19 11:37 ` Mike Jones 2022-08-22 15:49 ` zimoun
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).