From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pjotr Prins Subject: [Orchestration][RFC] A simple draft for channels Date: Mon, 19 Mar 2018 13:04:00 +0100 Message-ID: <20180319120400.GA13807@thebird.nl> References: <87bmhq6ytg.fsf@mdc-berlin.de> <87shar5wp8.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:39543) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1extaW-00016O-Uu for guix-devel@gnu.org; Mon, 19 Mar 2018 08:08:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1extaQ-0000WD-PQ for guix-devel@gnu.org; Mon, 19 Mar 2018 08:08:28 -0400 Received: from mail.thebird.nl ([95.154.246.10]:55897) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1extaQ-0000TH-J4 for guix-devel@gnu.org; Mon, 19 Mar 2018 08:08:22 -0400 Content-Disposition: inline In-Reply-To: <87shar5wp8.fsf@gmail.com> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Chris Marusich Cc: guix-devel@gnu.org, Ricardo Wurmus Let's start up again on this conversation in the context of deployment. I have a simple use case. For GeneNetwork we maintain GUIX_PACKAGE_PATH packages. It contains packages that ought to go in main line (such as python-gunicorn), but also packages that will never make it (such as a javascript twitter feed). Now we deploy multiple setups, which I'll simplify to development, testing and production here (there are more!). Each of these has a combination of a Guix git checkout and a GUIX_PACKAGE_PATH checkout. These git repo's are supposedly in sync with each other. Moving from one to the other, however, is too complicated and error prone. I can do it, but no one else really wants to. Even with my explanations it proves to be a royal pain. Especially with multiple people developing. The other thing is that it takes too long to rebuild the Guix repo. Even on 20+ cores we have to wait a significant amount of time. And things go wrong... Otherwise, only good stuff. I can provide binary packages, that is great. It is just this git juggling and Guix repo building that is killing me. Maybe the short term solution for us is to no longer use GUIX_PACKAGE_PATH, but to merge the differences in branches of the GNU Guix tree. That takes care of the syncing problem (though it will still be a headache). Maybe channels ought to be Guix git trees anyway, so that is one step in the right direction. Now I need a way to no longer rebuild all .go files for Guix tree updates/changes. Not only between switching branches, but also when just running 'git pull' from Guix savannah. I find I have to do that very often. So often that I don't even try running make anymore without make clean. Anyone here share that experience? One thing I could do is split out 3 git repos for every use case and update these individually not triggering rebuilds. And when I deploy on other machines move the complete repo across with .go files. Maybe also stick it into a container so I can deploy all dependencies with it and update any substitute keys in user land. HEY. Did I just invent a channel? If we allow channels to be architecture specific we could ship dependencies and .go files with the git tree. No rebuilds. If we allow using containers we can update substitute keys for channels (like mine on http://guix.genenetwork.org). The key updates are minor though. Main thing is to speed up deployment and make it less error prone. Maybe we should start thinking that a channel is simply an architecture dependent Guix 'pack' of substitutes that includes the pre-built Guix git repo. When deployed in a container we can inject the keys. When this works we can design a pack repository, making the channels searchable. Pj.