* Two copies of same library in store after cleaning generations and gc
@ 2022-08-14 11:31 Mike Jones
2022-08-18 12:13 ` Hartmut Goebel
0 siblings, 1 reply; 7+ messages in thread
From: Mike Jones @ 2022-08-14 11:31 UTC (permalink / raw)
To: help-guix
Hi Guix!
I've been running Guix System for a few weeks now, and I've noticed that
I have two copies of some libraries in my store (e.g. mesa, gtk) even
after updating all packages, deleting old profile generations, and
running gc.
This isn't a problem, as I have plenty of disk space, but I'm just
curious to understand why this happens.
For example, I have both
/gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8
/gnu/store/wj1nipayc74yalfmdg7wwimkfc1macbl-mesa-21.3.8
Using 'guix gc --referrers', I can see that:
- The first copy is referred to by all applications in my profile that
link against mesa, e.g. icecat.
- The second copy is referred to directly by my profile. Indeed, I can
see that, for example, ~/.guix-profile/lib/libGL.so is a symlink to
/gnu/store/wj1nipayc74yalfmdg7wwimkfc1macbl-mesa-21.3.8/lib/libGL.so.
So my question boils down to: why is the copy of mesa in
~/.guix-profile/lib different to the one to which programs in the same
profile are linked? And is there any guix command I can use to inspect
exactly how/why these copies differ?
Thanks!
- Mike
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc
2022-08-14 11:31 Two copies of same library in store after cleaning generations and gc Mike Jones
@ 2022-08-18 12:13 ` Hartmut Goebel
2022-08-18 20:30 ` Mike Jones
0 siblings, 1 reply; 7+ messages in thread
From: Hartmut Goebel @ 2022-08-18 12:13 UTC (permalink / raw)
To: Mike Jones, help-guix
Am 14.08.22 um 13:31 schrieb Mike Jones:
> So my question boils down to: why is the copy of mesa in
> ~/.guix-profile/lib different to the one to which programs in the same
> profile are linked? And is there any guix command I can use to inspect
> exactly how/why these copies differ?
I assume you installed mesa into your profile — or some other package
installed in your profile has mesa as „propagated input“.
Try „guix package -l“ to see whether it is installed in your profile.
--
Regards
Hartmut Goebel
| Hartmut Goebel |h.goebel@crazy-compilers.com |
|www.crazy-compilers.com | compilers which you thought are impossible |
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc
2022-08-18 12:13 ` Hartmut Goebel
@ 2022-08-18 20:30 ` Mike Jones
2022-08-19 6:13 ` Ricardo Wurmus
0 siblings, 1 reply; 7+ messages in thread
From: Mike Jones @ 2022-08-18 20:30 UTC (permalink / raw)
To: Hartmut Goebel, guix.rc7ys--- via
Hello,
Thanks for the hints.
> I assume you installed mesa into your profile — or some other package
> installed in your profile has mesa as „propagated input“.
>
> Try „guix package -l“ to see whether it is installed in your profile.
I haven't installed mesa explicitly, and it doesn't show up in `guix
package -l`, so I think you must be right that it's coming from a
propagated input.
I still wonder, though, why there are two different builds of mesa
appearing. I suspected it might be related to grafting, so I poked
around in /gnu/store and found the .drv files that produced these two
different copies. They both seem to be the result of a grafting process.
If I rebuild my profile with --no-grafts, the mesa used by programs and
the one in the profile are the same, so I'll just put this down to some
weirdness with grafting.
- Mike
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc
2022-08-18 20:30 ` Mike Jones
@ 2022-08-19 6:13 ` Ricardo Wurmus
2022-08-19 9:06 ` zimoun
0 siblings, 1 reply; 7+ messages in thread
From: Ricardo Wurmus @ 2022-08-19 6:13 UTC (permalink / raw)
To: Mike Jones; +Cc: Hartmut Goebel, help-guix
"Mike Jones" <mike@mjones.io> writes:
> If I rebuild my profile with --no-grafts, the mesa used by programs and
> the one in the profile are the same, so I'll just put this down to some
> weirdness with grafting.
This sounds like grafts working as intended. Do you also have more than
one instance of icecat – one using the graft and one without?
(Due to deduplication in /gnu/store this is not as wasteful as it may
seem.)
--
Ricardo
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc
2022-08-19 6:13 ` Ricardo Wurmus
@ 2022-08-19 9:06 ` zimoun
2022-08-19 11:37 ` Mike Jones
0 siblings, 1 reply; 7+ messages in thread
From: zimoun @ 2022-08-19 9:06 UTC (permalink / raw)
To: Ricardo Wurmus, Mike Jones; +Cc: Hartmut Goebel, help-guix
Hi,
On ven., 19 août 2022 at 08:13, Ricardo Wurmus <rekado@elephly.net> wrote:
> This sounds like grafts working as intended. Do you also have more than
> one instance of icecat – one using the graft and one without?
>
> (Due to deduplication in /gnu/store this is not as wasteful as it may
> seem.)
Using Guix 8f0d45c,
--8<---------------cut here---------------start------------->8---
$ guix build mesa
/gnu/store/5y054ijr0dqpd6v4wlby9alhbzmha033-mesa-21.3.8-bin
/gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8
$ guix build mesa --no-grafts
/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin
/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8
$ guix gc --references $(guix build icecat) | grep mesa
/gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8
--8<---------------cut here---------------end--------------->8---
it can be confusing… Let examine. :-)
First, let consider mesa referred by icecat.
--8<---------------cut here---------------start------------->8---
$ guix gc --derivers /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8
/gnu/store/qqa192m2cx4qrlvhr1dg7vrc49jqa84h-mesa-21.3.8.drv
/gnu/store/y2yxw4zd4p916lby3sibmskdv860zvpa-mesa-21.3.8.drv
--8<---------------cut here---------------end--------------->8---
It means that the both derivations,
/gnu/store/qqa192m2cx4qrlvhr1dg7vrc49jqa84h-mesa-21.3.8.drv
/gnu/store/y2yxw4zd4p916lby3sibmskdv860zvpa-mesa-21.3.8.drv
have the same output:
/gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8
Yes, that’s surprising but possible.
From /gnu/store/qqa192m2cx4qrlvhr1dg7vrc49jqa84h-mesa-21.3.8.drv, the
guile-builder reads:
--8<---------------cut here---------------start------------->8---
(define %build-inputs
(quote
(("x" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8")
[...]
((old-outputs
(quote
(("out" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8"))))
--8<---------------cut here---------------end--------------->8---
which grafts, I guess. Note that this derivation lists the other
derivation /gnu/store/0yd0gxbcrkh32zy4q8dkpyyv2xpvm1mh-mesa-21.3.8.drv.
From /gnu/store/y2yxw4zd4p916lby3sibmskdv860zvpa-mesa-21.3.8.drv, the
guile-builder is exactly the same (/gnu/store/a0h343…-mesa-21.3.8-guile-builder).
However, the derivation lists
/gnu/store/ylx0p9sl6qla4qsmp6dsvaqgd7g3m1rw-mesa-21.3.8.drv.
These two derivations,
/gnu/store/0yd0gxbcrkh32zy4q8dkpyyv2xpvm1mh-mesa-21.3.8.drv
/gnu/store/ylx0p9sl6qla4qsmp6dsvaqgd7g3m1rw-mesa-21.3.8.drv.
have the same outputs; which reads:
--8<---------------cut here---------------start------------->8---
Derive
([("bin","/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin","","")
,("out","/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8","","")]
--8<---------------cut here---------------end--------------->8---
other said, the ungrafted version. So far, so good!
Second, let consider the grafted build,
--8<---------------cut here---------------start------------->8---
$ guix gc --derivers /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8
/gnu/store/ji6h6y5jizwhsx2jzw67myk5fhiv87va-mesa-21.3.8.drv
--8<---------------cut here---------------end--------------->8---
and the guile-builder (/gnu/store/j4j58b…-mesa-21.3.8-guile-builder)
contains,
--8<---------------cut here---------------start------------->8---
(define %build-inputs
(quote
(("x" . "/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin")
("x" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8")
[...]
((old-outputs
(quote
(("bin" . "/gnu/store/znvm7i9rgaviwi2841j8gimanif99wl1-mesa-21.3.8-bin")
("out" . "/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8"))))
--8<---------------cut here---------------end--------------->8---
and that’s why the hash 74ags388lb… (from “guix build mesa”) is
different from lcqz4q383… (from icecat).
This derivation /gnu/store/ji6h6y5jizwhsx2jzw67myk5fhiv87va-mesa-21.3.8.drv
lists the other derivation,
/gnu/store/0yd0gxbcrkh32zy4q8dkpyyv2xpvm1mh-mesa-21.3.8.drv. Wait, it
has already appeared below. :-)
Well, all seems consistent, isn’t it? I hope now it is less confusing.
However, something still appears to me confusing:
--8<---------------cut here---------------start------------->8---
$ diff -r --no-dereference /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8
Binary files /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8/lib/dri/i830_dri.so and /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8/lib/dri/i830_dri.so differ
[...]
diff -r --no-dereference /gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8/lib/pkgconfig/dri.pc /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8/lib/pkgconfig/dri.pc
1c1
< prefix=/gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8
---
> prefix=/gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8
[...]
--8<---------------cut here---------------end--------------->8---
Other said, the grafted item is duplicated. Right?
Therefore, my question is: is deduplication optimal? Or better worded:
what is the waste by the duplicated grafted items?
Cheers,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc
2022-08-19 9:06 ` zimoun
@ 2022-08-19 11:37 ` Mike Jones
2022-08-22 15:49 ` zimoun
0 siblings, 1 reply; 7+ messages in thread
From: Mike Jones @ 2022-08-19 11:37 UTC (permalink / raw)
To: zimoun, Ricardo Wurmus; +Cc: Hartmut Goebel, guix.rc7ys--- via
Hi all!
On Fri, 19 Aug 2022, at 7:13 AM, Ricardo Wurmus wrote:
> This sounds like grafts working as intended. Do you also have more
> than one instance of icecat – one using the graft and one without?
After building the profile do I indeed have two instances of icecat in
the store. But when I run `guix gc`, the ungrafted one is of course
deleted, and I'm left with a single instance.
With mesa, however, there are always two active instances of mesa (also
some other libraries, e.g. gtk) referenced by the profile. I expected
that there would only be one (grafted) instance.
On Fri, 19 Aug 2022, at 10:06 AM, zimoun wrote:
> Well, all seems consistent, isn’t it? I hope now it is less
> confusing.
Yeah, thanks for working through this - I understand better now :) So we
have one original ungrafted output, which is:
/gnu/store/wdm2s2si8fqsrcd5xpc29ivmpkf20s8d-mesa-21.3.8
This then gets separately grafted twice to produce two new outputs:
/gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8 (result of `guix build mesa`)
/gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 (referred to by icecat)
But I wonder why icecat refers to a different mesa to `guix build
mesa`.
Running
$ diffoscope /gnu/store/lcqz4q3834bjd3dlc8zsr95mvzz9n006-mesa-21.3.8 \
/gnu/store/74ags388lb0xasf7v1h8j9ywj8zrsgv9-mesa-21.3.8`
(takes quite a long time to run!) shows me that many of the files are
different, but only in self-references.
I also diffed the guile-builder scripts for the two grafted mesa
instances. Here's the first difference:
- (map (lambda (o) (cons o (getenv o)))
- '("bin" "out")))
+ (map (lambda (o) (cons o (getenv o))) '("out")))
It seems that one is a graft of just the default output, while the other
also includes the "bin" output! I think this explains why there are two
separate grafts, that produce the same (minus self-references) output.
Presumably these aren't deduplicated due to the differing
self-references?
- Mike
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Two copies of same library in store after cleaning generations and gc
2022-08-19 11:37 ` Mike Jones
@ 2022-08-22 15:49 ` zimoun
0 siblings, 0 replies; 7+ messages in thread
From: zimoun @ 2022-08-22 15:49 UTC (permalink / raw)
To: Mike Jones, Ricardo Wurmus; +Cc: Hartmut Goebel, guix.rc7ys--- via
Hi,
On ven., 19 août 2022 at 12:37, "Mike Jones" <mike@mjones.io> wrote:
> Presumably these aren't deduplicated due to the differing
> self-references?
Yes.
Note that this happens for only the packages with multi-outputs impacted
by grafted ones. If I am correct, using Guix 65cabb0, the number of
packages with multi-outputs is 681 (see below).
On this number, a subset will be impacted by the ~11 grafted packages.
Cheers,
simon
--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use(ice-9 match)
scheme@(guix-user)> ,use(gnu)
scheme@(guix-user)> ,use(guix)
scheme@(guix-user)> (length (fold-packages
(lambda (package result)
(match (package-outputs package)
((x y z ...) (cons package result))
(_ result)))
'()))
$1 = 681
scheme@(guix-user)> (length (fold-packages
(lambda (package result)
(if (package-replacement package)
(cons package result)
result))
'()))
$2 = 11
--8<---------------cut here---------------end--------------->8---
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-08-22 18:44 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-14 11:31 Two copies of same library in store after cleaning generations and gc Mike Jones
2022-08-18 12:13 ` Hartmut Goebel
2022-08-18 20:30 ` Mike Jones
2022-08-19 6:13 ` Ricardo Wurmus
2022-08-19 9:06 ` zimoun
2022-08-19 11:37 ` Mike Jones
2022-08-22 15:49 ` zimoun
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).