unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#55499: excessively large manifests due to propagation
@ 2022-05-18 13:53 Ricardo Wurmus
  2022-05-24 15:30 ` Ludovic Courtès
  2022-05-31 16:09 ` bug#55499: [PATCH 0/3] Make 'manifest' files more compact Ludovic Courtès
  0 siblings, 2 replies; 11+ messages in thread
From: Ricardo Wurmus @ 2022-05-18 13:53 UTC (permalink / raw)
  To: 55499; +Cc: Ludovic Courtès

Packages of some languages rely heavily on propagation.  R is one
example.  Since the generated “manifest” file of a Guix profile records
entries for all propagated packages, this can get really big really
quickly.

A profile consisting only of four R packages (r-seurat, r-cistopic,
r-monocle3, and r-cicero-monocle3) results in a “manifest” file that
weighs 7.1MB.  At the MDC I repeatedly encountered manifest files that
are exceeding 24MB.

Simply reading that big a file with

    (call-with-input-file "huge-manifest" read)

takes several seconds.  On the MDC cluster I observed a delay of about
27 seconds.

Disabling read positions with (read-disable 'positions) significantly
speeds this up (18s vs 27s), but it’s still very slow.

We may be able to speed things up by supporting definitions or
references in manifest files, so that we don’t need to repeat a sub-tree
when generating the file.

-- 
Ricardo




^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2022-07-01 21:55 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-18 13:53 bug#55499: excessively large manifests due to propagation Ricardo Wurmus
2022-05-24 15:30 ` Ludovic Courtès
2022-05-25  4:35   ` Ricardo Wurmus
2022-05-31 16:09 ` bug#55499: [PATCH 0/3] Make 'manifest' files more compact Ludovic Courtès
2022-05-31 16:09   ` bug#55499: [PATCH 1/3] tests: Augment profile collision test Ludovic Courtès
2022-05-31 16:09   ` bug#55499: [PATCH 2/3] profiles: Do not repeat entries in 'manifest' file Ludovic Courtès
2022-05-31 17:35     ` Maxime Devos
2022-06-01  9:38       ` Ludovic Courtès
2022-05-31 16:09   ` bug#55499: [PATCH 3/3] squash! profiles: Make all entry fields optional Ludovic Courtès
2022-06-14  7:06   ` bug#55499: excessively large manifests due to propagation Ludovic Courtès
2022-07-01 21:54     ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).