Memoization profiling

Memoization profiling

[Ludovic Courtès]

Hello Guix,

Memoization plays a critical role in the performance of Guix.  I’ve
finally added support to profile memoization, which should allow us to
better understand how it performs and to use it more wisely:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=252c4083779a488c86e74362b4f3bb4bf927cc67
  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=6c80641d54a95f2da95e480a4a746761d25161e9

Here’s an example of what it gives (you need to “make clean-go &&
make”):

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=memoization ./pre-inst-env guix build libreoffice -d --no-grafts
/gnu/store/x2dk4zwkfvlypmk7havzhk71bgrlbfk3-libreoffice-5.3.7.2.drv
Memoization: 113 tables, 34 non-empty
  guix/utils.scm:744:2: 	6274 entries, 7645 lookups, 18% hits
  guix/derivations.scm:582:2: 	1942 entries, 5230 lookups, 63% hits
  guix/derivations.scm:636:2: 	1779 entries, 5066 lookups, 65% hits
  guix/derivations.scm:629:2: 	889 entries, 24116 lookups, 96% hits
  guix/packages.scm:776:4: 	710 entries, 770 lookups, 8% hits
  gnu/packages/bootstrap.scm:136:2: 	307 entries, 7634 lookups, 96% hits
  guix/build-system/asdf.scm:164:4: 	20 entries, 20 lookups, 0% hits
  guix/build-system/asdf.scm:164:4: 	11 entries, 38 lookups, 71% hits
  guix/packages.scm:776:4: 	10 entries, 10 lookups, 0% hits
  guix/derivations.scm:1028:2: 	8 entries, 5116 lookups, 100% hits
  guix/packages.scm:776:4: 	7 entries, 7 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	7 entries, 26 lookups, 73% hits
  guix/packages.scm:776:4: 	4 entries, 4 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	4 entries, 8 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	3 entries, 6 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	2 entries, 4 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	2 entries, 4 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	2 entries, 4 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	2 entries, 4 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/packages.scm:776:4: 	1 entries, 1 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 1 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 1 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 4 lookups, 75% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 7 lookups, 86% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 1 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 1 lookups, 0% hits
  guix/build-system/gnu.scm:88:6: 	1 entries, 2 lookups, 50% hits
--8<---------------cut here---------------end--------------->8---

Here memoization stats are dumped when exiting, but you can also force
them to be displayed after each GC:

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=memoization GUIX_PROFILING_EVENTS=after-gc \
  ./pre-inst-env guix build libreoffice -d --no-grafts
--8<---------------cut here---------------end--------------->8---

The new (guix profiling) module provides basic support for this.

Ludo’.