From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark Meyer Subject: Re: [orchestration] AWS public cloud images Date: Tue, 27 Mar 2018 23:33:35 +0200 Message-ID: <1522186415.2087803.1318240416.2A6F0401@webmail.messagingengine.com> References: <1522095505.3286349.1316797816.594CB0C5@webmail.messagingengine.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45641) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f0wDt-00046n-1J for guix-devel@gnu.org; Tue, 27 Mar 2018 17:33:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f0wDp-000507-Qk for guix-devel@gnu.org; Tue, 27 Mar 2018 17:33:41 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:54021) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f0wDp-0004zJ-M4 for guix-devel@gnu.org; Tue, 27 Mar 2018 17:33:37 -0400 In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: "Thompson, David" Cc: guix-devel Hi David, I think most of the AWS build processes feel imperative and hacky. I always equate this to a big pot of soup which you stir and pass on to the next guy, who builds another image/stirs it again. In general I think you should not start with an AMI, but with an empty volume and then deploy to that. Your AMI definition process should be composable, so if you have different teams working on it, i.e. systems engineering builds the base AMI and an application group customizes it, you should only run one AMI build and not pass down the artifact. To put it more extreme: it looks like some junkies passing around a dirty needle. That's one of the reasons the code to build the AMIs is currently the main deliverable. My proposition is not to refrain from providing a Golden Master image, it will be worthwhile for some purposes, my suggestion is to make the AMI build process fully automated and easily extensible. For the moment I don't mind having Packer in the mix. I don't however hold it in high regards, I think it's essentially glue code. But when you want to replace it, please consider that there are use cases like different partitions (GovCloud and AWS China) that have very different implementations of the AWS API. Speaking from my experience these can lead to huge headaches, and in general I think it's not worth to work around the same set of bugs another time. Cheers, Mark On Tue, Mar 27, 2018, at 20:08, Thompson, David wrote: > Hi Mark, > > On Mon, Mar 26, 2018 at 4:18 PM, Mark Meyer wrote: > > Hi, > > I've the beginning of Guix cloud images available over at Github at > > > > https://github.com/ofosos/guix-packer/ > > > > There's a small writeup of what has been done and what's still missing over here: > > > > https://ofosos.org/2018/03/26/guix-images-01/ > > > > All in all, I split the heavy lifting between Packer (AWS API calls) and `guix system` and that worked remarkably well. There's some integration with EC2 (You can inject a pubkey into the image via the console), but there's also a lot of stuff missing. In the end, I would like to provide public cloud images in some weeks time. Of course you'll have all the tools to rebuild your own images (surprisingly simple). > > > > I think there's still a lot of polish we can apply, but at some point we'll need some help from AWS. I do have AWS support access at work, but am not really comfortable to use company resources for this yet, but I'll probably ask if there's some avenue to get some 'official' help as a free software project, when I run across our technical account manager. > > First of all: Thanks! This is a great start! I've wanted to run GuixSD > EC2 instances for some time but haven't gotten around to it. The more > I think about it, though, the more I wonder how useful an official > GuixSD image is vs. providing software to create AMIs for any given > system configuration. The use-case I'm particularly interested in is > using GuixSD instances in auto scaling groups. I'm wondering if > there's any other way to create GuixSD AMIs than starting from some > official image, updating Guix, running 'guix system reconfigure', and > using the result as the basis for the AMI. While this ought to work, > it feels very imperative and hacky. In the future it would also be > great to eliminate the need for Packer entirely and replace it with > Guile. > > Anyway, just some food for thought. Awesome work! > > - Dave -- Mark Meyer mark@ofosos.org