Re: [Orchestration][RFC] A simple draft for channels

[ng0 <ng0@n0.is>]

Pjotr Prins transcribed 2.7K bytes:
> Let's start up again on this conversation in the context of
> deployment. I have a simple use case. For GeneNetwork we maintain
> GUIX_PACKAGE_PATH packages. It contains packages that ought to go in
> main line (such as python-gunicorn), but also packages that will never
> make it (such as a javascript twitter feed).
> 
> Now we deploy multiple setups, which I'll simplify to development,
> testing and production here (there are more!). Each of these has a
> combination of a Guix git checkout and a GUIX_PACKAGE_PATH checkout.

This reads a bit like what I'm working on and experimenting with
indepently at the moment.
I'm still waiting on what the turnout of this channels RFC will be.

So: I have a combination of guix.git on some-commit + a couple of
package-path repositories (put into classes "ports" and "pkgs")
and another couple of repositories which include other data.
All of this is used to construct a specified variant of an operating
system. Updates happen with a simple command so far, nothing's worked
out in a way that can be exposed to public. GUIX_PACKAGE_PATH for me
locally is super long by now. Essentially parts that never make their
way into Guix for various reasons are kept in modular pieces outside
of Guix (there's more to it, but that would be derailing the thread).

> These git repo's are supposedly in sync with each other.
> 
> Moving from one to the other, however, is too complicated and error
> prone. I can do it, but no one else really wants to. Even with my
> explanations it proves to be a royal pain.
> 
> Especially with multiple people developing.

> The other thing is that it
> takes too long to rebuild the Guix repo. Even on 20+ cores we have to
> wait a significant amount of time. And things go wrong... 

Yep. This is not really good in the very long term.

> Otherwise, only good stuff. I can provide binary packages, that is
> great. It is just this git juggling and Guix repo building that is
> killing me.

Didn't we talk about building Guix itself on the build farm? What's
keeping us from doing so?

> Maybe the short term solution for us is to no longer use
> GUIX_PACKAGE_PATH, but to merge the differences in branches of the GNU
> Guix tree. That takes care of the syncing problem (though it will
> still be a headache). Maybe channels ought to be Guix git trees
> anyway, so that is one step in the right direction.

I think this (just guix trees) is not really good, at least for my goals.
I'd rather have a modular layout where you can define multiple repositories
and channels are an addition to Guix. Unless we have good reasons to remove
the current extension-like behavior I'd like to keep them.

> Now I need a way to no longer rebuild all .go files for Guix tree
> updates/changes. Not only between switching branches, but also when
> just running 'git pull' from Guix savannah. I find I have to do that
> very often. So often that I don't even try running make anymore
> without make clean. Anyone here share that experience?
> 
> One thing I could do is split out 3 git repos for every use case and
> update these individually not triggering rebuilds. And when I deploy
> on other machines move the complete repo across with .go files.
> 
> Maybe also stick it into a container so I can deploy all dependencies
> with it and update any substitute keys in user land.
> 
> HEY. Did I just invent a channel? 
> 
> If we allow channels to be architecture specific we could ship
> dependencies and .go files with the git tree. No rebuilds.
> 
> If we allow using containers we can update substitute keys for
> channels (like mine on http://guix.genenetwork.org).
> 
> The key updates are minor though. Main thing is to speed up deployment
> and make it less error prone.
> 
> Maybe we should start thinking that a channel is simply an
> architecture dependent Guix 'pack' of substitutes that includes the
> pre-built Guix git repo. When deployed in a container we can inject
> the keys. When this works we can design a pack repository, making the
> channels searchable.
> 
> Pj.
> 
> 
> 

-- 
A88C8ADD129828D7EAC02E52E22F9BBFEE348588
https://n0.is