bug#63852: ‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

bug#63852: ‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

[Ludovic Courtès]

Consider this:

--8<---------------cut here---------------start------------->8---
$ cat /tmp/cran.scm
(cons (channel
       (name 'guix-cran)
       (url "https://github.com/guix-science/guix-cran.git"))
      %default-channels)
$ guix time-machine -C /tmp/cran.scm -- describe
Updating channel 'guix-cran' from Git repository at 'https://github.com/guix-science/guix-cran.git'...
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

[...]

The following derivations will be built:
  /gnu/store/90qiphp9048zgvi96lj8zqg8zhwi4js0-profile.drv
  /gnu/store/2lv56bl8ghyf2kvfkwj5iy3mrlfik4la-inferior-script.scm.drv
  /gnu/store/vnjf8gf6yf5gjm1syvz7yyfn71mmvy27-profile.drv
  /gnu/store/8x49757i1s0r6igrah8ramn9zc1v577b-guix-69dfdb7bd.drv
  /gnu/store/ll5x27npc3znyqv0rkjh945x9cr41k9q-guix-command.drv
  /gnu/store/m03hp1n62bv8gkid53g8igizbw012gai-guix-module-union.drv
  /gnu/store/pw8k2jwwyj2nwvqjcd1w3xjvz2n5zpbg-guix-69dfdb7bd-modules.drv
  /gnu/store/a6si5zgzsxyha2466v4q45hqimg31mjg-guix-config-modules.drv
  /gnu/store/3pm8aivzg3hfv95zlffxjrxvf8hfbg5n-guix-config-source.drv
  /gnu/store/rffsvls17s21v2h6gw7agah5vczwi3rm-config.scm.drv
  /gnu/store/v61jlksbgfslh27y8dbaj1jcajxz5c8z-guix-config.drv
  /gnu/store/z6r6xdcq9dyxlk8ln8cq1ka2gspxp83i-guix-daemon.drv
  /gnu/store/fshds3i961v1p3x8f7awh55lix0f7vkl-guix-cran.drv
  /gnu/store/xsnp3znb0h630kipjxd198qfd3jr2ww5-inferior-script.scm.drv

building /gnu/store/xsnp3znb0h630kipjxd198qfd3jr2ww5-inferior-script.scm.drv...
building /gnu/store/rffsvls17s21v2h6gw7agah5vczwi3rm-config.scm.drv...
building /gnu/store/3pm8aivzg3hfv95zlffxjrxvf8hfbg5n-guix-config-source.drv...
building /gnu/store/v61jlksbgfslh27y8dbaj1jcajxz5c8z-guix-config.drv...
building /gnu/store/a6si5zgzsxyha2466v4q45hqimg31mjg-guix-config-modules.drv...
building /gnu/store/pw8k2jwwyj2nwvqjcd1w3xjvz2n5zpbg-guix-69dfdb7bd-modules.drv...
building /gnu/store/m03hp1n62bv8gkid53g8igizbw012gai-guix-module-union.drv...
building /gnu/store/ll5x27npc3znyqv0rkjh945x9cr41k9q-guix-command.drv...
building /gnu/store/z6r6xdcq9dyxlk8ln8cq1ka2gspxp83i-guix-daemon.drv...
building /gnu/store/8x49757i1s0r6igrah8ramn9zc1v577b-guix-69dfdb7bd.drv...
building /gnu/store/fshds3i961v1p3x8f7awh55lix0f7vkl-guix-cran.drv...
building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 2 packages...
building /gnu/store/2lv56bl8ghyf2kvfkwj5iy3mrlfik4la-inferior-script.scm.drv...
building package cache...
--8<---------------cut here---------------end--------------->8---

It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
consumes memory until it gets OOM-killed.

It used to work fine a few months ago.

Ludo’.

bug#63852: ‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 5306 bytes --]

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
> consumes memory until it gets OOM-killed.

The culprit is easily found (here on just the 22K packages of ‘guix’, so
less than half of what you get with ‘guix-cran’):

--8<---------------cut here---------------start------------->8---
$ guix repl
GNU Guile 3.0.9
Copyright (C) 1995-2023 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,m(gnu packages)
scheme@(gnu packages)> ,use(statprof)
scheme@(gnu packages)> (gcprof (lambda () (generate-package-cache "/tmp/t")))
%     cumulative   self             
time   seconds     seconds  procedure
 50.00     13.77     13.77  ice-9/boot-9.scm:2498:4
  6.94      1.91      1.91  display
  5.56      1.53      1.53  object->string
  2.78      0.76      0.76  reverse
  2.78      0.76      0.76  gensym
  2.78      0.76      0.76  ice-9/boot-9.scm:2217:0:%load-announce
  1.39      6.88      0.38  gnu/packages.scm:388:4:expand-cache
  1.39      0.76      0.38  srfi/srfi-1.scm:1028:0:lset-intersection
  1.39      0.38      0.38  number->string
  1.39      0.38      0.38  gnu/packages/crates-io.scm:17783:2:arguments
  1.39      0.38      0.38  open-output-string
  1.39      0.38      0.38  gnu/packages/crates-io.scm:18450:2:arguments
  1.39      0.38      0.38  gnu/packages/crates-io.scm:36208:2:arguments
  1.39      0.38      0.38  guix/packages.scm:1320:8:mproc
  1.39      0.38      0.38  gnu/packages/crates-io.scm:25101:2:arguments
  1.39      0.38      0.38  gnu/packages/crates-io.scm:29972:2:arguments
  1.39      0.38      0.38  gnu/packages/crates-io.scm:883:2:arguments
  1.39      0.38      0.38  ice-9/vlist.scm:449:0:vhash-cons
  1.39      0.38      0.38  append
  1.39      0.38      0.38  string-append
  1.39      0.38      0.38  system/vm/assembler.scm:2175:8
  1.39      0.38      0.38  guix/build/syscalls.scm:1168:0:read-dirent-header/linux
  1.39      0.38      0.38  srfi/srfi-1.scm:1033:17
  1.39      0.38      0.38  gnu/packages/crates-io.scm:9336:2:arguments
  1.39      0.38      0.38  gnu/packages/crates-io.scm:64009:2:arguments
  1.39      0.38      0.38  guix/build-system/cargo.scm:229:4
  1.39      0.38      0.38  ice-9/boot-9.scm:2759:0:module-make-local-var!
  0.00   7328.64      0.00  system/vm/assembler.scm:1258:0:intern-constant
  0.00   1423.97      0.00  ice-9/boot-9.scm:220:5:map1
  0.00    201.90      0.00  ice-9/threads.scm:388:4

[...]

---
Sample count: 72
Total time: 27.531145665 seconds (14.056589389 seconds in GC)
--8<---------------cut here---------------end--------------->8---

The assembler allocates a huge constant table in ‘intern-constant’
(‘generate-package-cache’ produces one literal list containing one small
vector per package, each of which contains strings and other constants).

The attached file emulates what ‘generate-package-cache’ does.  Each
vector in the list looks like this:

  #(0 "xyzxyz-0" "1.0.0" (gnu packages xyz0) xyzxyz-0 ("out") #t #f "gnu/packages/xyz0" 0 1)

The program terminates with roughly a 250 MiB heap for 22K entries—about
10 MiB per vector.

The heap profile on completion looks like this:

--8<---------------cut here---------------start------------->8---
  %   type                               self    avg obj size
 57.5 bytevector                          8,205,632 455868.4
 23.9 vector                              3,413,857   549.3
  7.7 struct                              1,093,840    40.8
  5.3 pair                                  753,376    16.0
  2.7 stringbuf                             385,856    51.1
  1.1 symbol                                151,424    32.0
  0.4 program                                59,424    47.3
  0.3 string                                 49,152    32.0
  0.2 heap-number                            23,520    32.0
  0.2 variable                               21,696    30.3
  0.1 smob                                   21,056    32.0
  0.1 vm-continuation                        20,864    32.0
  0.1 atomic-box                             20,736    32.0
  0.1 weak-table                             19,584    31.8
  0.1 unknown                                19,040    32.0
  0.0 pointer                                 2,000    16.0
  0.0 hash-table                                896    32.0
  0.0 primitive                                 144    16.0
  0.0 weak-vector                                96    16.0
  0.0 frame                                      96    32.0
  0.0 primitive-generic                          64    32.0
  0.0 keyword                                    16    16.0
sampled heap: 13.60166 MiB (heap size: 233.16016 MiB)
--8<---------------cut here---------------end--------------->8---

It’s likely that the ‘buf’ field of <asm> is what’s taking space

On that topic, see also:

  https://lists.gnu.org/archive/html/guile-devel/2023-01/msg00013.html

Ludo’.


[-- Attachment #2: the reproducer --]
[-- Type: text/plain, Size: 1673 bytes --]

;; https://issues.guix.gnu.org/63852

(use-modules (system base compile)
             (language tree-il)
             (srfi srfi-1)
             (srfi srfi-26)
             (ice-9 match)
             (ice-9 time)
             (statprof))

(define lst
  (unfold (cut > <> 22000)
          (lambda (i)
            ;; Mimic 'generate-package-cache'.
            (define name
              (string-append "xyzxyz-" (number->string i 8)))
            (vector i name
                    (string-append "1.0."
                                   (number->string (modulo i 300)))
                    (list 'gnu 'packages
                          (string->symbol
                           (string-append "xyz" (number->string i))))
                    (string->symbol name)
                    '("out")
                    #t
                    #f
                    (string-append "gnu/packages/xyz"
                                   (number->string (modulo i 300)))
                    (modulo i 1000)
                    1))
          1+
          0))

(pk 'go! (gc-stats))
(let ((exp (match (or (getenv "VARIANT") "0")
             ("0" `',lst)
             ("1" `(list ,@lst))
             ("2" (make-const #f lst))            ;avoid psyntax overhead
             ("3" (make-primcall #f 'list
                                 (map (lambda (vector)
                                        (make-const #f vector))
                                      lst))))))
  (compile exp #:to 'bytecode
           #:from (if (pair? exp)
                      'scheme
                      'tree-il)
           #:optimization-level 0
           #:opts '(#:to-file? #t)))
(pk (gc-stats))

bug#63852: ‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
>> consumes memory until it gets OOM-killed.
>
> The culprit is easily found (here on just the 22K packages of ‘guix’, so
> less than half of what you get with ‘guix-cran’):

(Beware of “easily” when the guy doesn’t know what he’s talking about.)

Turns out the problem is more pedestrian: ‘r-future-tests’ depends on
itself.

--8<---------------cut here---------------start------------->8---
$ guix import cran future.tests

Starting download of /tmp/guix-file.TOBjcf
From http://cran.r-project.org/src/contrib/future.tests_0.7.0.tar.gz...
 …_0.7.0.tar.gz  186KiB               1.2MiB/s 00:00 ▕██████████████████▏ 100.0%
(package
  (name "r-future-tests")
  (version "0.7.0")
  (source (origin
            (method url-fetch)
            (uri (cran-uri "future.tests" version))
            (sha256
             (base32
              "10g1w99xqr7l0cn27642aphqcvfidgpas38f84r815yy6k1ryrlx"))))
  (properties `((upstream-name . "future.tests")))
  (build-system r-build-system)
  (propagated-inputs (list r-cli r-crayon r-future r-prettyunits r-sessioninfo))
  (native-inputs (list r-future-tests))
  (home-page "https://future.tests.futureverse.org")
  (synopsis "Test Suite for 'Future API' Backends")
  (description
   "Backends implementing the Future API, as defined by the future package, should
use the tests provided by this package to validate that they meet the minimal
requirements of the Future API. The tests can be performed easily from within R
or from outside of R from the command line making it straightforward to include
them in package tests and in Continuous Integration (CI) pipelines.")
$ guix describe
Generation 265  Jun 04 2023 23:48:31    (current)
  guix eed55a6
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea
--8<---------------cut here---------------end--------------->8---

To be fixed in the importer.

Ludo’.

bug#63852: ‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>>
>>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
>>> consumes memory until it gets OOM-killed.
>>
>> The culprit is easily found (here on just the 22K packages of ‘guix’, so
>> less than half of what you get with ‘guix-cran’):
>
> (Beware of “easily” when the guy doesn’t know what he’s talking about.)
>
> Turns out the problem is more pedestrian: ‘r-future-tests’ depends on
> itself.

Here’s a patch to prevent that from happening again:

  https://issues.guix.gnu.org/63917

(We also need to fix the importer.)

Ludo’.

bug#63852: ‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>>
>>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
>>> consumes memory until it gets OOM-killed.
>>
>> The culprit is easily found (here on just the 22K packages of ‘guix’, so
>> less than half of what you get with ‘guix-cran’):
>
> (Beware of “easily” when the guy doesn’t know what he’s talking about.)
>
> Turns out the problem is more pedestrian: ‘r-future-tests’ depends on
> itself.

With commit e4259d4e9e3251e4c4b45d1cce4008ac32b504c8, such cycles are
detected early on.

Closing this bug.

We should still improve the memory footprint of the assembler though.

Ludo’.