[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

Hello!

The attached patches add ‘guix graph --path’, a command to display
the shortest path between two nodes:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph --path emacs libffi
emacs@26.3
gnutls@3.6.9
guile@2.2.6
libffi@3.2.1
$ ./pre-inst-env guix graph --path -t derivation emacs libffi
/gnu/store/aswcmllr300bbsiv1i63idpivzkzq2f2-emacs-26.3.drv
/gnu/store/dfcgalgx2fnc7alsi0qdjfzghn517ha3-libffi-3.2.1.drv
$ ./pre-inst-env guix graph --path -t references libreoffice llvm@9
/gnu/store/1rbww1g8q5sc9x3v318wp6xn62832n5m-libreoffice-6.4.2.2
/gnu/store/ir05kr2z31xgaih9k5z4xah7k3gqs0sk-libepoxy-1.5.4
/gnu/store/2mf0clz9w64diy0kz11qcs4q5wg9hc6z-mesa-19.3.4
/gnu/store/vsd496n5arjjlriqw914syirhyscq8q1-llvm-9.0.1
--8<---------------cut here---------------end--------------->8---

It was long overdue, and it’s rather cool.

There’s a bikeshedding opportunity in the last patch: should it go
in ‘guix graph’ or elsewhere?  I think ‘guix graph’ is a good home
for that, and could eventually include more graph queries.  For
instance, ‘guix refresh -l’ could very well live in ‘guix graph’.

Feedback welcome!

Ludo’.

Ludovic Courtès (3):
  graph: reference/referrer node types work with graph traversal.
  graph: Add 'shortest-path'.
  guix graph: Add '--path'.

 doc/guix.texi          | 48 ++++++++++++++++++++++-
 guix/graph.scm         | 69 ++++++++++++++++++++++++++++++++-
 guix/scripts/graph.scm | 69 +++++++++++++++++++++++++++------
 tests/graph.scm        | 88 ++++++++++++++++++++++++++++++++++++++++++
 tests/guix-graph.sh    | 16 +++++++-
 5 files changed, 274 insertions(+), 16 deletions(-)

-- 
2.26.2

[bug#41164] [PATCH 1/3] graph: reference/referrer node types work with graph traversal.

[Ludovic Courtès]

The graph traversal procedures in (guix graph) assume that nodes can be
compared with 'eq?', which was not the case for nodes of
%REFERENCE-NODE-TYPE and %REFERRER-NODE-TYPE (strings).

* guix/scripts/graph.scm (intern): New procedure.
(ensure-store-items, references*)
(%reference-node-type, non-derivation-referrers)
(%referrer-node-type): Use it on all store items.
* tests/graph.scm ("node-transitive-edges, references"): New test.
---
 guix/scripts/graph.scm | 23 ++++++++++++++++-------
 tests/graph.scm        | 27 +++++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 7 deletions(-)

diff --git a/guix/scripts/graph.scm b/guix/scripts/graph.scm
index fca1e3777c..d69dace14f 100644
--- a/guix/scripts/graph.scm
+++ b/guix/scripts/graph.scm
@@ -307,6 +307,14 @@ derivation graph")))))))
 ;;; DAG of residual references (aka. run-time dependencies).
 ;;;
 
+(define intern
+  (mlambda (str)
+    "Intern STR, a string denoting a store item."
+    ;; This is necessary for %REFERENCE-NODE-TYPE and %REFERRER-NODE-TYPE
+    ;; because their nodes are strings but the (guix graph) traversal
+    ;; procedures expect to be able to compare nodes with 'eq?'.
+    str))
+
 (define ensure-store-items
   ;; Return a list of store items as a monadic value based on the given
   ;; argument, which may be a store item or a package.
@@ -316,10 +324,10 @@ derivation graph")))))))
      (mlet %store-monad ((drv (package->derivation package)))
        (return (match (derivation->output-paths drv)
                  (((_ . file-names) ...)
-                  file-names)))))
+                  (map intern file-names))))))
     ((? store-path? item)
      (with-monad %store-monad
-       (return (list item))))
+       (return (list (intern item)))))
     (x
      (raise
       (condition (&message (message "unsupported argument for \
@@ -333,18 +341,19 @@ substitutes."
     (guard (c ((store-protocol-error? c)
                (match (substitutable-path-info store (list item))
                  ((info)
-                  (values (substitutable-references info) store))
+                  (values (map intern (substitutable-references info))
+                          store))
                  (()
                   (leave (G_ "references for '~a' are not known~%")
                          item)))))
-      (values (references store item) store))))
+      (values (map intern (references store item)) store))))
 
 (define %reference-node-type
   (node-type
    (name "references")
    (description "the DAG of run-time dependencies (store references)")
    (convert ensure-store-items)
-   (identifier (lift1 identity %store-monad))
+   (identifier (lift1 intern %store-monad))
    (label store-path-package-name)
    (edges references*)))
 
@@ -353,14 +362,14 @@ substitutes."
     (lambda (item)
       "Return the referrers of ITEM, except '.drv' files."
       (mlet %store-monad ((items (referrers item)))
-        (return (remove derivation-path? items))))))
+        (return (map intern (remove derivation-path? items)))))))
 
 (define %referrer-node-type
   (node-type
    (name "referrers")
    (description "the DAG of referrers in the store")
    (convert ensure-store-items)
-   (identifier (lift1 identity %store-monad))
+   (identifier (lift1 intern %store-monad))
    (label store-path-package-name)
    (edges non-derivation-referrers)))
 
diff --git a/tests/graph.scm b/tests/graph.scm
index 402847102f..983a6ed654 100644
--- a/tests/graph.scm
+++ b/tests/graph.scm
@@ -31,6 +31,7 @@
   #:use-module (guix utils)
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bootstrap)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages libunistring)
   #:use-module (gnu packages bootstrap)
@@ -358,6 +359,32 @@ edges."
         (return (lset= eq? (node-transitive-edges (list p2) edges)
                        (list p1a p1b p0)))))))
 
+(test-assert "node-transitive-edges, references"
+  (run-with-store %store
+    (mlet* %store-monad ((d0 (package->derivation %bootstrap-guile))
+                         (d1 (gexp->derivation "d1"
+                                               #~(begin
+                                                   (mkdir #$output)
+                                                   (symlink #$%bootstrap-guile
+                                                            (string-append
+                                                             #$output "/l")))))
+                         (d2 (gexp->derivation "d2"
+                                               #~(begin
+                                                   (mkdir #$output)
+                                                   (symlink #$d1
+                                                            (string-append
+                                                             #$output "/l")))))
+                         (_  (built-derivations (list d2)))
+                         (->node -> (node-type-convert %reference-node-type))
+                         (o2      (->node (derivation->output-path d2)))
+                         (o1      (->node (derivation->output-path d1)))
+                         (o0      (->node (derivation->output-path d0)))
+                         (edges   (node-edges %reference-node-type
+                                              (append o0 o1 o2)))
+                         (reqs    ((store-lift requisites) o2)))
+      (return (lset= string=?
+                     (append o2 (node-transitive-edges o2 edges)) reqs)))))
+
 (test-equal "node-reachable-count"
   '(3 3)
   (run-with-store %store
-- 
2.26.2

[bug#41164] [PATCH 2/3] graph: Add 'shortest-path'.

[Ludovic Courtès]

* guix/graph.scm (shortest-path): New procedure.
* tests/graph.scm ("shortest-path, packages + derivations")
("shortest-path, reverse packages")
("shortest-path, references"): New tests.
---
 guix/graph.scm  | 69 ++++++++++++++++++++++++++++++++++++++++++++++++-
 tests/graph.scm | 61 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 129 insertions(+), 1 deletion(-)

diff --git a/guix/graph.scm b/guix/graph.scm
index d7fd5f3e4b..b695ca4306 100644
--- a/guix/graph.scm
+++ b/guix/graph.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2015, 2016, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2016 Ricardo Wurmus <rekado@elephly.net>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -42,6 +42,7 @@
             traverse/depth-first
             node-transitive-edges
             node-reachable-count
+            shortest-path
 
             %graph-backends
             %d3js-backend
@@ -140,6 +141,72 @@ typically returned by 'node-edges' or 'node-back-edges'."
                         0
                         nodes node-edges))
 
+(define (shortest-path node1 node2 type)
+  "Return as a monadic value the shorted path, represented as a list, from
+NODE1 to NODE2 of the given TYPE.  Return #f when there is no path."
+  (define node-edges
+    (node-type-edges type))
+
+  (define (find-shortest lst)
+    ;; Return the shortest path among LST, where each path is represented as a
+    ;; vlist.
+    (let loop ((lst lst)
+               (best +inf.0)
+               (shortest #f))
+      (match lst
+        (()
+         shortest)
+        ((head . tail)
+         (let ((len (vlist-length head)))
+           (if (< len best)
+               (loop tail len head)
+               (loop tail best shortest)))))))
+
+  (define (find-path node path paths)
+    ;; Return the a vhash that maps nodes to paths, with each path from the
+    ;; given node to NODE2.
+    (define (augment-paths child paths)
+      ;; When using %REFERENCE-NODE-TYPE, nodes can contain self references,
+      ;; hence this test.
+      (if (eq? child node)
+          (store-return paths)
+          (find-path child vlist-null paths)))
+
+    (cond ((eq? node node2)
+           (store-return (vhash-consq node (vlist-cons node path)
+                                      paths)))
+          ((vhash-assq node paths)
+           (store-return paths))
+          (else
+           ;; XXX: We could stop recursing if one if CHILDREN is NODE2, but in
+           ;; practice it's good enough.
+           (mlet* %store-monad ((children (node-edges node))
+                                (paths    (foldm %store-monad
+                                                 augment-paths
+                                                 paths
+                                                 children)))
+             (define sub-paths
+               (filter-map (lambda (child)
+                             (match (vhash-assq child paths)
+                               (#f #f)
+                               ((_ . path) path)))
+                           children))
+
+             (match sub-paths
+               (()
+                (return (vhash-consq node #f paths)))
+               (lst
+                (return (vhash-consq node
+                                     (vlist-cons node (find-shortest sub-paths))
+                                     paths))))))))
+
+  (mlet %store-monad ((paths (find-path node1
+                                        (vlist-cons node1 vlist-null)
+                                        vlist-null)))
+    (return (match (vhash-assq node1 paths)
+              ((_ . #f) #f)
+              ((_ . path) (vlist->list path))))))
+
 \f
 ;;;
 ;;; Graphviz export.
diff --git a/tests/graph.scm b/tests/graph.scm
index 983a6ed654..136260c7d1 100644
--- a/tests/graph.scm
+++ b/tests/graph.scm
@@ -398,4 +398,65 @@ edges."
         (return (list (node-reachable-count (list p2) edges)
                       (node-reachable-count (list p0) back)))))))
 
+(test-equal "shortest-path, packages + derivations"
+  '(("p5" "p4" "p1" "p0")
+    ("p3" "p2" "p1" "p0")
+    #f
+    ("p5-0.drv" "p4-0.drv" "p1-0.drv" "p0-0.drv"))
+  (run-with-store %store
+    (let* ((p0 (dummy-package "p0"))
+           (p1 (dummy-package "p1" (inputs `(("p0" ,p0)))))
+           (p2 (dummy-package "p2" (inputs `(("p1" ,p1)))))
+           (p3 (dummy-package "p3" (inputs `(("p2" ,p2)))))
+           (p4 (dummy-package "p4" (inputs `(("p1" ,p1)))))
+           (p5 (dummy-package "p5" (inputs `(("p4" ,p4) ("p3" ,p3))))))
+      (mlet* %store-monad ((path1 (shortest-path p5 p0 %package-node-type))
+                           (path2 (shortest-path p3 p0 %package-node-type))
+                           (nope  (shortest-path p3 p4 %package-node-type))
+                           (drv5  (package->derivation p5))
+                           (drv0  (package->derivation p0))
+                           (path3 (shortest-path drv5 drv0
+                                                 %derivation-node-type)))
+        (return (append (map (lambda (path)
+                               (and path (map package-name path)))
+                             (list path1 path2 nope))
+                        (list (map (node-type-label %derivation-node-type)
+                                   path3))))))))
+
+(test-equal "shortest-path, reverse packages"
+  '("libffi" "guile" "guile-json")
+  (run-with-store %store
+    (mlet %store-monad ((path (shortest-path (specification->package "libffi")
+                                             guile-json
+                                             %reverse-package-node-type)))
+      (return (map package-name path)))))
+
+(test-equal "shortest-path, references"
+  `(("d2" "d1" ,(package-full-name %bootstrap-guile "-"))
+    (,(package-full-name %bootstrap-guile "-") "d1" "d2"))
+  (run-with-store %store
+    (mlet* %store-monad ((d0 (package->derivation %bootstrap-guile))
+                         (d1 (gexp->derivation "d1"
+                                               #~(begin
+                                                   (mkdir #$output)
+                                                   (symlink #$%bootstrap-guile
+                                                            (string-append
+                                                             #$output "/l")))))
+                         (d2 (gexp->derivation "d2"
+                                               #~(begin
+                                                   (mkdir #$output)
+                                                   (symlink #$d1
+                                                            (string-append
+                                                             #$output "/l")))))
+                         (_  (built-derivations (list d2)))
+                         (->node -> (node-type-convert %reference-node-type))
+                         (o2   (->node (derivation->output-path d2)))
+                         (o0   (->node (derivation->output-path d0)))
+                         (path (shortest-path (first o2) (first o0)
+                                              %reference-node-type))
+                         (rpath (shortest-path (first o0) (first o2)
+                                               %referrer-node-type)))
+      (return (list (map (node-type-label %reference-node-type) path)
+                    (map (node-type-label %referrer-node-type) rpath))))))
+
 (test-end "graph")
-- 
2.26.2

[bug#41164] [PATCH 3/3] guix graph: Add '--path'.

[Ludovic Courtès]

* guix/scripts/graph.scm (display-path): New procedure.
(%options, show-help): Add '--path'.
(guix-graph): Handle it.
* tests/guix-graph.sh: Add tests.
* doc/guix.texi (Invoking guix graph): Document it.
(Invoking guix size): Mention it.
---
 doc/guix.texi          | 48 ++++++++++++++++++++++++++++++++++++++++--
 guix/scripts/graph.scm | 46 +++++++++++++++++++++++++++++++++++-----
 tests/guix-graph.sh    | 16 +++++++++++++-
 3 files changed, 102 insertions(+), 8 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 2ed545847b..e174e13887 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -10021,6 +10021,12 @@ In this example we see that the combination of the four packages takes
 102.3@tie{}MiB in total, which is much less than the sum of each closure
 since they have a lot of dependencies in common.
 
+When looking at the profile returned by @command{guix size}, you may
+find yourself wondering why a given package shows up in the profile at
+all.  To understand it, you can use @command{guix graph --path -t
+references} to display the shortest path between the two packages
+(@pxref{Invoking guix graph}).
+
 The available options are:
 
 @table @option
@@ -10081,8 +10087,9 @@ directly to the @command{dot} command of Graphviz.  It can also emit an
 HTML page with embedded JavaScript code to display a ``chord diagram''
 in a Web browser, using the @uref{https://d3js.org/, d3.js} library, or
 emit Cypher queries to construct a graph in a graph database supporting
-the @uref{https://www.opencypher.org/, openCypher} query language.
-The general syntax is:
+the @uref{https://www.opencypher.org/, openCypher} query language.  With
+@option{--path}, it simply displays the shortest path between two
+packages.  The general syntax is:
 
 @example
 guix graph @var{options} @var{package}@dots{}
@@ -10228,6 +10235,29 @@ collected.
 
 @end table
 
+@cindex shortest path, between packages
+Often, the graph of the package you are interested in does not fit on
+your screen, and anyway all you want to know is @emph{why} that package
+actually depends on some seemingly unrelated package.  The
+@option{--path} option instructs @command{guix graph} to display the
+shortest path between two packages (or derivations, or store items,
+etc.):
+
+@example
+$ guix graph --path emacs libunistring
+emacs@@26.3
+mailutils@@3.9
+libunistring@@0.9.10
+$ guix graph --path -t derivation emacs libunistring
+/gnu/store/@dots{}-emacs-26.3.drv
+/gnu/store/@dots{}-mailutils-3.9.drv
+/gnu/store/@dots{}-libunistring-0.9.10.drv
+$ guix graph --path -t references emacs libunistring
+/gnu/store/@dots{}-emacs-26.3
+/gnu/store/@dots{}-libidn2-2.2.0
+/gnu/store/@dots{}-libunistring-0.9.10
+@end example
+
 The available options are the following:
 
 @table @option
@@ -10248,6 +10278,20 @@ List the supported graph backends.
 
 Currently, the available backends are Graphviz and d3.js.
 
+@item --path
+Display the shortest path between two nodes of the type specified by
+@option{--type}.  The example below shows the shortest path between
+@code{libreoffice} and @code{llvm} according to the references of
+@code{libreoffice}:
+
+@example
+$ guix graph --path -t references libreoffice llvm
+/gnu/store/@dots{}-libreoffice-6.4.2.2
+/gnu/store/@dots{}-libepoxy-1.5.4
+/gnu/store/@dots{}-mesa-19.3.4
+/gnu/store/@dots{}-llvm-9.0.1
+@end example
+
 @item --expression=@var{expr}
 @itemx -e @var{expr}
 Consider the package @var{expr} evaluates to.
diff --git a/guix/scripts/graph.scm b/guix/scripts/graph.scm
index d69dace14f..1d5db3b3cb 100644
--- a/guix/scripts/graph.scm
+++ b/guix/scripts/graph.scm
@@ -455,6 +455,29 @@ package modules, while attempting to retain user package modules."
                       (graph-backend-description backend)))
             %graph-backends))
 
+\f
+;;;
+;;; Displaying a path.
+;;;
+
+(define (display-path node1 node2 type)
+  "Display the shortest path from NODE1 to NODE2, of TYPE."
+  (mlet %store-monad ((path (shortest-path node1 node2 type)))
+    (define node-label
+      (let ((label (node-type-label type)))
+        ;; Special-case derivations and store items to print them in full,
+        ;; contrary to what their 'node-type-label' normally does.
+        (match-lambda
+          ((? derivation? drv) (derivation-file-name drv))
+          ((? string? str) str)
+          (node (label node)))))
+
+    (if path
+        (format #t "~{~a~%~}" (map node-label path))
+        (leave (G_ "no path from '~a' to '~a'~%")
+               (node-label node1) (node-label node2)))
+    (return #t)))
+
 \f
 ;;;
 ;;; Command-line options.
@@ -465,6 +488,9 @@ package modules, while attempting to retain user package modules."
                  (lambda (opt name arg result)
                    (alist-cons 'node-type (lookup-node-type arg)
                                result)))
+         (option '("path") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'path? #t result)))
          (option '("list-types") #f #f
                  (lambda (opt name arg result)
                    (list-node-types)
@@ -510,6 +536,8 @@ Emit a representation of the dependency graph of PACKAGE...\n"))
   -t, --type=TYPE        represent nodes of the given TYPE"))
   (display (G_ "
       --list-types       list the available graph types"))
+  (display (G_ "
+      --path             display the shortest path between the given nodes"))
   (display (G_ "
   -e, --expression=EXPR  consider the package EXPR evaluates to"))
   (display (G_ "
@@ -566,11 +594,19 @@ Emit a representation of the dependency graph of PACKAGE...\n"))
           (mlet %store-monad ((_     (set-grafting #f))
                               (nodes (mapm %store-monad
                                            (node-type-convert type)
-                                           items)))
-            (export-graph (concatenate nodes)
-                          (current-output-port)
-                          #:node-type type
-                          #:backend backend))
+                                           (reverse items))))
+            (if (assoc-ref opts 'path?)
+                (match nodes
+                  (((node1 _ ...) (node2 _ ...))
+                   (display-path node1 node2 type))
+                  (_
+                   (leave (G_ "'--path' option requires exactly two \
+nodes (given ~a)~%")
+                          (length nodes))))
+                (export-graph (concatenate nodes)
+                              (current-output-port)
+                              #:node-type type
+                              #:backend backend)))
           #:system (assq-ref opts 'system)))))
   #t)
 
diff --git a/tests/guix-graph.sh b/tests/guix-graph.sh
index 4c37b61b38..ccb4933c88 100644
--- a/tests/guix-graph.sh
+++ b/tests/guix-graph.sh
@@ -1,5 +1,5 @@
 # GNU Guix --- Functional package management for GNU
-# Copyright © 2015, 2016, 2019 Ludovic Courtès <ludo@gnu.org>
+# Copyright © 2015, 2016, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
 # Copyright © 2019 Simon Tournier <zimon.toutoune@gmail.com>
 #
 # This file is part of GNU Guix.
@@ -82,3 +82,17 @@ then false; else true; fi
 
 # Try --load-path
 guix graph -L $module_dir dummy | grep 'label = "dummy'
+
+# Displaying shortest paths (or lack thereof).
+if guix graph --path emacs vim; then false; else true; fi
+
+path="\
+emacs
+gnutls
+guile
+libffi"
+test "`guix graph --path emacs libffi | cut -d '@' -f1`" = "$path"
+
+# At the derivation level, there's a direct path because libffi is propagated
+# via gtk+.
+test "`guix graph --path -t derivation emacs libffi | wc -l`" -ge 2
-- 
2.26.2

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[zimoun]

Hi Ludo,

Awesome!
I remember discussing such feature at FOSDEM. :-)
And a couple of days ago (updating with core-updates), I was annoyed
because a lot of "unexpected" packages were downloaded, I asked myself
"why".  So you removed one item of my feature wishlist. ;-)


On Sun, 10 May 2020 at 01:05, Ludovic Courtès <ludo@gnu.org> wrote:

> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix graph --path emacs libffi
> --8<---------------cut here---------------end--------------->8---

Well, the command is obviously not symmetric (oriented).  The path is
from 'emacs' to 'libffi'.

It appears to me not clear in '--help'.

--8<---------------cut here---------------start------------->8---
display the shortest path between the given nodes
--8<---------------cut here---------------end-------------->8---

Well, I am already bikeshedding but the CLI "guix graph emacs
--path-to libffi" appears to me clearer (or '--why' as "why" is
mentioned and underlined in the doc :-)).


> There’s a bikeshedding opportunity in the last patch: should it go
> in ‘guix graph’ or elsewhere?  I think ‘guix graph’ is a good home
> for that, and could eventually include more graph queries.  For
> instance, ‘guix refresh -l’ could very well live in ‘guix graph’.

I do not have a strong opinion.  The "graph" subcommand fits well.
But for example:

  guix show emacs --why libffi
  guix search emacs --why libffi

make sense too -- at least to me. ;-)


And bikeshedding again, there is a temptation  to pipe the current CLI:

  guix graph --path emacs libffi | guix show

which does not obviously work because of "show".  From my point of
view, this CLI seems good:

    guix search emacs --why libffi | guix show --format=oneline

emacs@26.3 The extensible, customizable, self-documenting text editor
gnutls@3.6.9 Transport layer security library
guile@3.0.2 Scheme implementation intended especially for extensions
libffi@3.3 Foreign function call interface library

And '--format' could be 'recutils' or whatever.  But that another story. :-)


Last, it is not new but confusing, the node "guile@3.0.2" is returned
but it is reachable with "guile-next".


All the best,
simon

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> I remember discussing such feature at FOSDEM. :-)
> And a couple of days ago (updating with core-updates), I was annoyed
> because a lot of "unexpected" packages were downloaded, I asked myself
> "why".  So you removed one item of my feature wishlist. ;-)

Yeah, I guess we’re all periodically annoyed by that.  :-)

> On Sun, 10 May 2020 at 01:05, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> --8<---------------cut here---------------start------------->8---
>> $ ./pre-inst-env guix graph --path emacs libffi
>> --8<---------------cut here---------------end--------------->8---
>
> Well, the command is obviously not symmetric (oriented).  The path is
> from 'emacs' to 'libffi'.
>
> It appears to me not clear in '--help'.
>
> display the shortest path between the given nodes

‘--help’ is not a substitute for the manual though.  Do you have another
wording proposal in mind that is as concise?

> Well, I am already bikeshedding but the CLI "guix graph emacs
> --path-to libffi" appears to me clearer (or '--why' as "why" is
> mentioned and underlined in the doc :-)).

Actually, as things are, the location of ‘--path’ on the command line
does not matter at all.  I don’t think it’s worth changing.

As for ‘--path-to’ instead of ‘--path’: I think a path in common
language is directed (one follows a path from one place to another).

>> There’s a bikeshedding opportunity in the last patch: should it go
>> in ‘guix graph’ or elsewhere?  I think ‘guix graph’ is a good home
>> for that, and could eventually include more graph queries.  For
>> instance, ‘guix refresh -l’ could very well live in ‘guix graph’.
>
> I do not have a strong opinion.

Good!  I knew there’d be candidates for the bikeshedding opportunity.
:-)

> The "graph" subcommand fits well.  But for example:
>
>   guix show emacs --why libffi
>   guix search emacs --why libffi
>
> make sense too -- at least to me. ;-)

Dunno, I’m not convinced.

> And bikeshedding again, there is a temptation  to pipe the current CLI:
>
>   guix graph --path emacs libffi | guix show

‘guix graph --path emacs libffi | xargs guix show’

Though ‘guix show’ appears to ignore all but one argument, uh.

Thanks!

Ludo’.

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[zimoun]

Hi Ludo,

Reproducing the example from the manual leads to this error:

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix graph --path -t references libreoffice llvm
guix graph: error: no path from
'/gnu/store/rd24gl402vy5g5sm3azhq13mwdg5fbi1-libreoffice-6.4.2.2' to
'/gnu/store/bcgnx704v408sr7lcy480wcnbkcixwrw-llvm-10.0.0-opt-viewer'
--8<---------------cut here---------------end--------------->8---


On Sun, 10 May 2020 at 16:16, Ludovic Courtès <ludo@gnu.org> wrote:

> > It appears to me not clear in '--help'.
> >
> > display the shortest path between the given nodes
>
> ‘--help’ is not a substitute for the manual though.  Do you have another

Yes for sure.  '--help' is only a remind.  I mean I use it like that:
I know something but I do not remember the exact details on how to use
it (option name, arguments order, etc.)


> wording proposal in mind that is as concise?

display the shortest path from node1 to node2


> As for ‘--path-to’ instead of ‘--path’: I think a path in common
> language is directed (one follows a path from one place to another).

My english is probably not enough good.  (Note that is the same for me
with the french word "chemin".)
To me, "path" is often ambiguous because sometimes it means oriented
and sometimes not; depending on the context: topology, graph theory,
common life: synonymous of road or track and synonymous of route or
direction.

This ambiguity added to the non positional option '--path' appears to
me confusing from where to where.

My 2 remarks come from this ambiguity of the word "path".
Well, maybe it is only me.


> > The "graph" subcommand fits well.  But for example:
> >
> >   guix show emacs --why libffi
> >   guix search emacs --why libffi
> >
> > make sense too -- at least to me. ;-)
>
> Dunno, I’m not convinced.

The main issue with "search" is that the subcommand accepts an
regxexps and so it breaks consistency.  Therefore, I answer to myself:
it is a bad idea. :-)


> > And bikeshedding again, there is a temptation  to pipe the current CLI:
> >
> >   guix graph --path emacs libffi | guix show
>
> ‘guix graph --path emacs libffi | xargs guix show’
>
> Though ‘guix show’ appears to ignore all but one argument, uh.

Yes.  Because it is a pure alias of "guix package --show='.


Cheers,
simon

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[zimoun]

On Sun, 10 May 2020 at 18:18, zimoun <zimon.toutoune@gmail.com> wrote:

> > ‘guix graph --path emacs libffi | xargs guix show’
> >
> > Though ‘guix show’ appears to ignore all but one argument, uh.
>
> Yes.  Because it is a pure alias of "guix package --show='.

To be more explicit, it is because 'process-query' processes only one
query.  Which makes sense in the context of "guix package", e.g.,
"guix package --show=emacs --search=libffi" will be a bad idea, IMHO.
BUt not necessary "guix package --show=emacs --show=libffi".  I do not
know,.

There is an easy fix in 'guix/scripts/show.scm':

   (map (lambda opt (guix-package* opt)) (reverse opts))

but I am thinking if something better is possible to fix "--show=emacs
--show=libffi" too.

Cheers,
simon

[bug#41164] Fix pipe 'guix show'

[zimoun]

Hi Ludo,

> ‘guix graph --path emacs libffi | xargs guix show’
>
> Though ‘guix show’ appears to ignore all but one argument, uh.

Just to let you know that the patch [1] should fix the issue.  Now
"--show" is processed as "--search" and so piping works.

[1] issues.guix.gnu.org/41183

Cheers,
simon

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

zimoun <zimon.toutoune@gmail.com> skribis:

> On Sun, 10 May 2020 at 18:18, zimoun <zimon.toutoune@gmail.com> wrote:
>
>> > ‘guix graph --path emacs libffi | xargs guix show’
>> >
>> > Though ‘guix show’ appears to ignore all but one argument, uh.
>>
>> Yes.  Because it is a pure alias of "guix package --show='.
>
> To be more explicit, it is because 'process-query' processes only one
> query.  Which makes sense in the context of "guix package", e.g.,
> "guix package --show=emacs --search=libffi" will be a bad idea, IMHO.
> BUt not necessary "guix package --show=emacs --show=libffi".  I do not
> know,.
>
> There is an easy fix in 'guix/scripts/show.scm':
>
>    (map (lambda opt (guix-package* opt)) (reverse opts))
>
> but I am thinking if something better is possible to fix "--show=emacs
> --show=libffi" too.

Let’s discuss it and above all fix it in a separate thread!  :-)

Ludo’.

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> ./pre-inst-env guix graph --path -t references libreoffice llvm
> guix graph: error: no path from
> '/gnu/store/rd24gl402vy5g5sm3azhq13mwdg5fbi1-libreoffice-6.4.2.2' to
> '/gnu/store/bcgnx704v408sr7lcy480wcnbkcixwrw-llvm-10.0.0-opt-viewer'

There’s a path to llvm@9 though.

Such examples will of course bitrot over time because they depend on the
dependency graph, which keeps changing.  But that’s OK IMO: the goal is
just to give an idea of what the tool can do.

>> wording proposal in mind that is as concise?
>
> display the shortest path from node1 to node2

‘node1’ and ‘node2’ are undefined though…

>> As for ‘--path-to’ instead of ‘--path’: I think a path in common
>> language is directed (one follows a path from one place to another).
>
> My english is probably not enough good.  (Note that is the same for me
> with the french word "chemin".)
> To me, "path" is often ambiguous because sometimes it means oriented
> and sometimes not; depending on the context: topology, graph theory,
> common life: synonymous of road or track and synonymous of route or
> direction.
>
> This ambiguity added to the non positional option '--path' appears to
> me confusing from where to where.
>
> My 2 remarks come from this ambiguity of the word "path".
> Well, maybe it is only me.

I don’t know, but I’m still unconvinced that ‘--path-to’ is any better
than ‘--path’.  In the end, the intended audience consists of people who
know we’re talking about a DAG.

Thanks for your feedback!

Ludo’.

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[zimoun]

On Mon, 11 May 2020 at 14:36, Ludovic Courtès <ludo@gnu.org> wrote:

> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > ./pre-inst-env guix graph --path -t references libreoffice llvm
> > guix graph: error: no path from
> > '/gnu/store/rd24gl402vy5g5sm3azhq13mwdg5fbi1-libreoffice-6.4.2.2' to
> > '/gnu/store/bcgnx704v408sr7lcy480wcnbkcixwrw-llvm-10.0.0-opt-viewer'
>
> There’s a path to llvm@9 though.

Well, I do not know if the issue is not about the "outputs" 'out' vs
'opt-viewer'.  I probably do not understand how 'references' works but
'dbus' is an 'inputs' of 'vlc' and:

  ./pre-inst-env guix graph --path vlc -t references dbus
guix graph: error: no path from
'/gnu/store/2qk6h97bhm5zii59005ws5qk6drhdp3b-vlc-3.0.10' to
'/gnu/store/6a65in084gfn5bbgyna179irhghvqfpy-dbus-1.12.16-doc'



Cheers,
simon

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Mon, 11 May 2020 at 14:36, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> zimoun <zimon.toutoune@gmail.com> skribis:
>>
>> > ./pre-inst-env guix graph --path -t references libreoffice llvm
>> > guix graph: error: no path from
>> > '/gnu/store/rd24gl402vy5g5sm3azhq13mwdg5fbi1-libreoffice-6.4.2.2' to
>> > '/gnu/store/bcgnx704v408sr7lcy480wcnbkcixwrw-llvm-10.0.0-opt-viewer'
>>
>> There’s a path to llvm@9 though.
>
> Well, I do not know if the issue is not about the "outputs" 'out' vs
> 'opt-viewer'.  I probably do not understand how 'references' works but
> 'dbus' is an 'inputs' of 'vlc' and:
>
>   ./pre-inst-env guix graph --path vlc -t references dbus
> guix graph: error: no path from
> '/gnu/store/2qk6h97bhm5zii59005ws5qk6drhdp3b-vlc-3.0.10' to
> '/gnu/store/6a65in084gfn5bbgyna179irhghvqfpy-dbus-1.12.16-doc'

You’re doing it right, but the ‘dbus’ package has two outputs and the
thing just picks the first one, which is not the one you’re interested
in.

I’ll check if we can just arrange to parse outputs instead of packages.

Thanks,
Ludo’.

bug#41164: [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

I pushed the whole series:

  88a96c568c guix graph: Add '--path'.
  36c2192414 graph: Add 'shortest-path'.
  7240202136 graph: reference/referrer node types work with graph traversal.

Ludo’.

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[zimoun]

Hi Ludo,

On Mon, 11 May 2020 at 22:55, Ludovic Courtès <ludo@gnu.org> wrote:

> You’re doing it right, but the ‘dbus’ package has two outputs and the
> thing just picks the first one, which is not the one you’re interested
> in.

Yes, my point is: the example "libreoffice --path llvm" does not work
because of that.  Not because llvm@9 vs llvm@10, I guess. :-)

I have tried to find another example without several outputs but did
not find one complex enough to be "interesting"; that's why the most
simple 'vlc'  and 'dbus'. :-)
There is always an output (doc or debug or etc.) which breaks '--path
-t references'.

Well, I am not sure to understand how it works for 'references'
because I miss how several outputs are managed: same key, two values.
And from my understanding, the last value (doc or debug or etc. and
generally not out) replaces any other.


Cheers,
simon

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[Ludovic Courtès]

Hello,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Mon, 11 May 2020 at 22:55, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> You’re doing it right, but the ‘dbus’ package has two outputs and the
>> thing just picks the first one, which is not the one you’re interested
>> in.
>
> Yes, my point is: the example "libreoffice --path llvm" does not work
> because of that.  Not because llvm@9 vs llvm@10, I guess. :-)

It works for me:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph --path libreoffice llvm@9
libreoffice@6.4.2.2
glew@2.1.0
mesa@19.3.4
llvm@9.0.1
--8<---------------cut here---------------end--------------->8---

> I have tried to find another example without several outputs but did
> not find one complex enough to be "interesting"; that's why the most
> simple 'vlc'  and 'dbus'. :-)
> There is always an output (doc or debug or etc.) which breaks '--path
> -t references'.

Note that ‘-t references’ accept store file names, and that’s in fact
the main use case.  So you can always do:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph --path -t references vlc $(guix build dbus --no-grafts |grep [0-9]$)
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
/gnu/store/hmxvyis9d6qzpbdy0dj0fqq2b1dz4swb-vlc-3.0.10
/gnu/store/9xbmwzl08wpkdjvya4x55hksi134255l-dbus-1.12.16
--8<---------------cut here---------------end--------------->8---

But the nice thing with ‘-t references’ is that you don’t need to have
the thing available on disk:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph -t references --path vim tcsh
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
/gnu/store/b7hhq1l0ykw70yl360phlkcy3074rvwf-vim-8.2.0411
/gnu/store/yym79jarrj2x3bbsnalvgb0pim2xbm2m-tcsh-6.22.02
$ ls /gnu/store/b7hhq1l0ykw70yl360phlkcy3074rvwf-vim-8.2.0411
ls: ne eblas atingi '/gnu/store/b7hhq1l0ykw70yl360phlkcy3074rvwf-vim-8.2.0411': Dosiero aŭ dosierujo ne ekzistas
--8<---------------cut here---------------end--------------->8---

Ludo’.

[bug#41164] [PATCH 0/3] Add 'guix graph --path'

[zimoun]

Hi Ludo,

On Tue, 12 May 2020 at 10:41, Ludovic Courtès <ludo@gnu.org> wrote:

> But the nice thing with ‘-t references’ is that you don’t need to have
> the thing available on disk:
>
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix graph -t references --path vim tcsh
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> /gnu/store/b7hhq1l0ykw70yl360phlkcy3074rvwf-vim-8.2.0411
> /gnu/store/yym79jarrj2x3bbsnalvgb0pim2xbm2m-tcsh-6.22.02
> $ ls /gnu/store/b7hhq1l0ykw70yl360phlkcy3074rvwf-vim-8.2.0411
> ls: ne eblas atingi '/gnu/store/b7hhq1l0ykw70yl360phlkcy3074rvwf-vim-8.2.0411': Dosiero aŭ dosierujo ne ekzistas
> --8<---------------cut here---------------end--------------->8---

Mitigate by the several outputs issues. :-)
Well, there is small room for a fix: "-t reference --path node1 node2"
where by default 'nodeX' refers to the output 'out', otherwise
specified by 'nodeX:output'.


Thank you for all the explanations.  Really cool new feature!

Cheers,
simon