‘cargo-build-system’ makes everything slow

["Ludovic Courtès" <ludovic.courtes@inria.fr>]

[-- Attachment #1: Type: text/plain, Size: 2329 bytes --]

Hi,

I was profiling the ‘dependents’ procedure use by the new ‘guix build
--dependents’ option in (guix scripts build).  Like ‘guix refresh -l’,
it lowers all the packages to a bag, constructs the graph, and traverses
all the nodes backward.  And it’s slow (~25s).

The profile is clear (test program below):

--8<---------------cut here---------------start------------->8---
100.0% with-exception-handler at ice-9/boot-9.scm:1689:4
  100.0% thunk at guix/store.scm:686:4
    100.0% run-with-store at guix/store.scm:2196:0
      100.0% #x7f1cb0ccbeb0 at guix/scripts/build.scm:596:6
        100.0% #x7f1c9c353ef0 at guix/graph.scm:98:7
          99.6% #x7f1cb1b61e00 at guix/store.scm:1925:0
            99.6% loop at guix/store.scm:1925:0
              99.6% add-edge at guix/graph.scm:92:7
                99.6% #x7f1c9c3340bc at guix/scripts/graph.scm:187:10
                  96.3% #x15d3b3bc
                    95.8% bag-node-edges at guix/scripts/graph.scm:159:0
                      93.1% package->bag at guix/packages.scm:1755:0
                        92.5% thunk at guix/packages.scm:1763:4
                          89.3% lower at guix/build-system/cargo.scm:305:0
                            33.5% vhash-assq at ice-9/vlist.scm:539:0
                              9.4% #x15d3cf98
                            21.0% #x15d3de90
                            3.6% vhash-cons at ice-9/vlist.scm:449:0
                              0.9% #x15d3cf98
                              0.1% #x15d3b430
                                0.1% #x15d3daf4
                                  0.1% reap-pipes at ice-9/popen.scm:183:0
                            2.5% filter-map at srfi/srfi-1.scm:681:0
                              2.0% #x7f1cb0a997f4 at guix/build-system/cargo.scm:298:4
--8<---------------cut here---------------end--------------->8---

89% of the time is spent lowering ‘cargo-build-system’ packages,
specifically in ‘expand-crate-sources’, which computes more or less the
same thing over and over again.  (It also returns more than just
transitive sources of Crates, AFAICS.)

Problem is little can be done about it, at least not without triggering
a world rebuild.

But even then, it would help a lot to just not do that.

Thoughts?

Ludo’.


[-- Attachment #2: test program --]
[-- Type: text/plain, Size: 422 bytes --]

(use-modules (statprof)
             (ice-9 time)
             (guix)
             ((gnu packages) #:select (all-packages))
             ((guix scripts build) #:select (dependents))
             (gnu packages version-control))

(time (->bool (all-packages)))                    ;warm up

(with-store store
  (statprof
   (lambda ()
     (dependents store (list libgit2) 1))
   #:full-stacks? #t
   #:display-style 'tree))