bug#30710: guix graph gives duplicate nodes

bug#30710: guix graph gives duplicate nodes

[Hartmut Goebel]

Hi,

"guix graph" delivers the same package with different IDs. Here is an
example with a node delivered twice. (For plasma-workspace, which I'm
working on, this package was even listed four times).

$ ./pre-inst-env guix graph -t package -b graphviz qtbase | grep
autoconf-wrapper
  "59511552" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "59511744" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

bug#30710: guix graph gives duplicate nodes

[Ludovic Courtès]

Hello,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> "guix graph" delivers the same package with different IDs. Here is an
> example with a node delivered twice. (For plasma-workspace, which I'm
> working on, this package was even listed four times).
>
> $ ./pre-inst-env guix graph -t package -b graphviz qtbase | grep
> autoconf-wrapper
>   "59511552" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "59511744" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];

This is expected.  Strictly speaking, we’re talking about two different
package objects, hence the different IDs.

Now, there are cases were we have multiple package objects mapping to a
single derivation.  That’s OK.  You can use “guix graph -t bag” or
similar if you want nodes that correspond to derivations.

HTH!

Ludo’.

bug#30710: guix graph gives duplicate nodes

[Hartmut Goebel]

Hi,
> This is expected.  Strictly speaking, we’re talking about two different
> package objects, hence the different IDs.

I wonder

a) whether it is useful to have different graph nodes for the same package.

This is about usability of the resulting graph, esp. since this is the
default graph output format. Does it help *users* for their analysis? Or
is this some *expert* insight?

b) how there can be different package objects for the same package

To my understanding, e.g. (gnu packages base) is loaded once, defining
package object abcd once and assigning it to a variable. All packages
referring to abcd use the some package object. So there should be only
*one* package object.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

bug#30710: guix graph gives duplicate nodes

[Ludovic Courtès]

Hello,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

>> This is expected.  Strictly speaking, we’re talking about two different
>> package objects, hence the different IDs.
>
> I wonder
>
> a) whether it is useful to have different graph nodes for the same package.
>
> This is about usability of the resulting graph, esp. since this is the
> default graph output format. Does it help *users* for their analysis? Or
> is this some *expert* insight?

As explained in “Invoking guix graph”, the tool provides different graph
types, each at its own abstraction level.

The package graph is high-level, but as a side-effect it has this
artifact.

To developers it’s actually useful to see the graph of package objects.
There are cases where, with functions that return packages, you would
notice that you’re generating lots of package objects for the same
underlying derivation, which is super inefficient (in particular it
defeats memoization).

Most of the time, there’s exactly one package object for each
derivation; if not, that’s usually a bug.

> b) how there can be different package objects for the same package
>
> To my understanding, e.g. (gnu packages base) is loaded once, defining
> package object abcd once and assigning it to a variable. All packages
> referring to abcd use the some package object. So there should be only
> *one* package object.

(eq? foo (package (inherit foo))) => #false

Yet, these two packages map to the very same derivation.

HTH,
Ludo’.

bug#30710: guix graph gives duplicate nodes

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 3556 bytes --]

On Wed, 07 Mar 2018 16:18:51 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Hello,
> 
> Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:
> 
> >> This is expected.  Strictly speaking, we’re talking about two
> >> different package objects, hence the different IDs.  
> >
> > I wonder
> >
> > a) whether it is useful to have different graph nodes for the same
> > package.
> >
> > This is about usability of the resulting graph, esp. since this is
> > the default graph output format. Does it help *users* for their
> > analysis? Or is this some *expert* insight?  
> 
> As explained in “Invoking guix graph”, the tool provides different
> graph types, each at its own abstraction level.
> 
> The package graph is high-level, but as a side-effect it has this
> artifact.
> 
> To developers it’s actually useful to see the graph of package
> objects. There are cases where, with functions that return packages,
> you would notice that you’re generating lots of package objects for
> the same underlying derivation, which is super inefficient (in
> particular it defeats memoization).
> 
> Most of the time, there’s exactly one package object for each
> derivation; if not, that’s usually a bug.
> 
> > b) how there can be different package objects for the same package
> >
> > To my understanding, e.g. (gnu packages base) is loaded once,
> > defining package object abcd once and assigning it to a variable.
> > All packages referring to abcd use the some package object. So
> > there should be only *one* package object.  
> 
> (eq? foo (package (inherit foo))) => #false
> 
> Yet, these two packages map to the very same derivation.


This thing really took me a while to think about the graph system in
general and this concrete case.

Speaking about this concrete case: If you inspect the output of 

`guix graph qt` you find these interesting lines:

  "64168128" [label = "autoconf-wrapper-2.69", shape = box, fontname = Helvetica
  "64167936" [label = "autoconf-wrapper-2.69", shape = box, fontname = Helvetica

  "52941184" -> "64168128" [color = darkseagreen];
  "52940800" -> "64167936" [color = blue];

  "52941184" [label = "automake-1.15.1", shape = box, fontname = Helvetica];
  "52940800" [label = "libtool-2.4.6", shape = box, fontname = Helvetica];


If you look into gnu/packages/autotools.scm, you see that
autoconf-wrapper is not a package, but a package-factory:

(define* (autoconf-wrapper #:optional (autoconf autoconf))

Now the package definitions of "automake" and "libtool" each use the
same fragment of code in their native-inputs, but a different "package"
in the eq?-sense, although they basically want the same thing:

 `(("autoconf" ,(autoconf-wrapper))

As ludo stated above: "Most of the time, there’s exactly one package
object for each derivation; if not, that’s usually a bug."

This looks to me like a bug.

Correction:

(define autoconf-wrapper-default (autoconf-wrapper))

And then use this singular package as native-inputs to libtool and automake.

Furthermore, when I search:

find . -name "*.scm" -exec grep -H "autoconf-wrapper" "{}" ";" | less

I find about 10 packages that use the fabrik, but all in the default way.

So instead of:

  #:export (autoconf-wrapper))


We could just 

(define-public autoconf-wrapper-default (autoconf-wrapper))

and use that.

Or, if noone is using this fabrik, just drop that and make a normal package out of it.

WDYT? Reopen this one?

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

bug#30710: guix graph gives duplicate nodes

[Ludovic Courtès]

Björn Höfling <bjoern.hoefling@bjoernhoefling.de> skribis:

> If you look into gnu/packages/autotools.scm, you see that
> autoconf-wrapper is not a package, but a package-factory:
>
> (define* (autoconf-wrapper #:optional (autoconf autoconf))
>
> Now the package definitions of "automake" and "libtool" each use the
> same fragment of code in their native-inputs, but a different "package"
> in the eq?-sense, although they basically want the same thing:
>
>  `(("autoconf" ,(autoconf-wrapper))
>
> As ludo stated above: "Most of the time, there’s exactly one package
> object for each derivation; if not, that’s usually a bug."
>
> This looks to me like a bug.
>
> Correction:
>
> (define autoconf-wrapper-default (autoconf-wrapper))
>
> And then use this singular package as native-inputs to libtool and automake.
>
> Furthermore, when I search:
>
> find . -name "*.scm" -exec grep -H "autoconf-wrapper" "{}" ";" | less
>
> I find about 10 packages that use the fabrik, but all in the default way.
>
> So instead of:
>
>   #:export (autoconf-wrapper))
>
>
> We could just 
>
> (define-public autoconf-wrapper-default (autoconf-wrapper))
>
> and use that.
>
> Or, if noone is using this fabrik, just drop that and make a normal package out of it.
>
> WDYT? Reopen this one?

Good catch!  I implemented what you suggest above in commit
464f5447396fcec9b43f7eab71d5d42b522a157f.

Thank you!

Ludo’.

bug#30710: guix graph gives duplicate nodes

[Hartmut Goebel]

On Fri, 09 Mar 2018 23:59:26 +0100 ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Good catch!  I implemented what you suggest above in commit
>> 464f5447396fcec9b43f7eab71d5d42b522a157f.

Thanks, this solved the issue only partially: On my "kde-plasma" branch:

$ ./pre-inst-env guix graph plasma-workspace | grep autoconf
  "133094208" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "133094976" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
  "152772352" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "152420544" [label = "autoconf-2.69", shape = box, fontname = Helvetica];

There are other packages, which are duplicate but don't have a factory
AFAIK:

$ ./pre-inst-env guix graph plasma-workspace | grep kbus
  "130257472" [label = "kdbusaddons-5.42.0", shape = box, fontname =
Helvetica];
  "148171200" [label = "kdbusaddons-5.42.0", shape = box, fontname =
Helvetica];

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

bug#30710: guix graph gives duplicate nodes

[Ludovic Courtès]

Hello,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> On Fri, 09 Mar 2018 23:59:26 +0100 ludo@gnu.org (Ludovic Courtès) wrote:
>>
>>> Good catch!  I implemented what you suggest above in commit
>>> 464f5447396fcec9b43f7eab71d5d42b522a157f.
>
> Thanks, this solved the issue only partially: On my "kde-plasma" branch:
>
> $ ./pre-inst-env guix graph plasma-workspace | grep autoconf
>   "133094208" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "133094976" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
>   "152772352" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "152420544" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
>
> There are other packages, which are duplicate but don't have a factory
> AFAIK:
>
> $ ./pre-inst-env guix graph plasma-workspace | grep kbus
>   "130257472" [label = "kdbusaddons-5.42.0", shape = box, fontname =
> Helvetica];
>   "148171200" [label = "kdbusaddons-5.42.0", shape = box, fontname =
> Helvetica];

That stems from the use of ‘package-mapping’ or
‘package-input-rewriting’ (maybe via ‘package-with-python2’), which can
create distinct package objects mapping to the same derivation.

I don’t have a good solution here.

Thanks,
Ludo’.