unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: jgart <jgart@dismail.de>
To: Guix Devel <guix-devel@gnu.org>
Subject: Caching The Python World: guix shell --container
Date: Sat, 15 Jan 2022 02:30:47 -0500	[thread overview]
Message-ID: <20220115023047.GB14101@gac.attlocal.net> (raw)

Hi Guixers, 

I've been developing an app[0] that launches jupyter notebooks in "the
cloud" in guix containers[1]. In other words, the user submits a url for a
git repo containing the jupyter notebook and the app runs the notebook
in a guix container serving it to the user's browser tab.

The app auto-detects a manifest.scm in the root of the jupyter notebook's
project repository. The app then builds the container and all the
dependencies for the notebook specified by the manifest.scm.

The issue is the following: 

Sometimes a manifest with 10 notebook dependencies or even less will take
a long time (> 3 minutes) for Guix to build the container with everything
specified by the manifest.

For example, this repository containing a guix manifest caused the
browser to timeout out today after minutes of waiting for guix to build
the container.

https://github.com/BonfaceKilz/tsaf-analysis-of-bxd-mouse-colonies/blob/main/manifest.scm

Is there a way to pre-cache all the guix packaged binaries or a
subset thereof *ahead of time* so that if the user needs a particular
package/library they can get it near instantaneously[2]/on demand when
the app calls guix shell?

In other words, I am interested in pre-caching "the python world"
packaged by GNU Guix upstream. 

Is there currently a convenient way to tell Guix to build "all python-*
packages" and cache them? One way that comes to mind is dumping all
the python packages to a manifest and regularly updating that manifest
as new packages are added by maintainers. 

Does this sound like a sane way to solve this container build waiting issue?

Any thoughts/advice is much appreciated.

all best,

jgart

[0]: https://git.genenetwork.org/jgart/binderlite
[1]: https://guix.gnu.org/en/blog/2015/container-provisioning-with-guix/
[2]: https://guix.gnu.org/en/blog/2021/from-guix-environment-to-guix-shell/
"hot cache, in 0.1 seconds"



             reply	other threads:[~2022-01-15  7:32 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-15  7:30 jgart [this message]
2022-01-15  7:45 ` Caching The Python World: guix shell --container Ricardo Wurmus
2022-01-15  8:24   ` jgart
2022-01-15 11:25     ` Ricardo Wurmus

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220115023047.GB14101@gac.attlocal.net \
    --to=jgart@dismail.de \
    --cc=guix-devel@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).