From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Re: [Mark Meyer] Re: AWS + OpenStack support Date: Tue, 11 Apr 2017 00:04:34 -0700 Message-ID: <87h91vctml.fsf@gmail.com> References: <871sszh4vj.fsf@ofosos.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:44773) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cxpqy-0002Zy-Ng for guix-devel@gnu.org; Tue, 11 Apr 2017 03:04:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cxpqx-0003ui-B5 for guix-devel@gnu.org; Tue, 11 Apr 2017 03:04:40 -0400 Received: from mail-pf0-x22b.google.com ([2607:f8b0:400e:c00::22b]:36085) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cxpqx-0003uS-3l for guix-devel@gnu.org; Tue, 11 Apr 2017 03:04:39 -0400 Received: by mail-pf0-x22b.google.com with SMTP id o126so44552919pfb.3 for ; Tue, 11 Apr 2017 00:04:38 -0700 (PDT) In-Reply-To: <871sszh4vj.fsf@ofosos.org> (Mark Meyer's message of "Tue, 11 Apr 2017 07:48:00 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Mark Meyer Cc: guix-devel@gnu.org --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Mark Meyer writes: > Chris> Long story short, instead of starting with a base image and > Chris> modifying it (e.g., by injecting credentials at first boot > Chris> via the EC2 metadata service), one appealing alternative is > Chris> to use EC2's VM import feature to actually import precisely > Chris> the system that you want to launch: > > Chris> https://aws.amazon.com/ec2/vm-import/ > > Which does not work with GuixSD (tried it). Apparently it looks into the > image an expects stuff like fstab. I find it not very trust building > that it actually inspects the image. Yes, we would need to write some code that creates the kind of image ( that EC2 expects. The documentation claims that "raw" format is supported; I guess they have some unspoken requirements (like the fstab thing you mentioned) that complicate matters? https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-impor= t.html#prerequisites-image For what it's worth, if you have a support contract (even a low tier one) with AWS, their support engineers can help answer questions. I'm sure they could also help figure out what the exact restrictions on the formats are, if the documentation is not clear enough: https://docs.aws.amazon.com/awssupport/latest/user/getting-started.html > Chris> Customizations, such as SSH credentials, would be specified > Chris> in a GuixSD operating system configuration file and built > Chris> into the VM image, so neither the EC2 metadata service, nor > Chris> hacks like the "cloud-init" script used by some distros, > Chris> would enter into the picture at all. > > Chris> Some preliminary work in a similar spirit was already done in > Chris> the branch 'wip-deploy', but I don't think it was > Chris> EC2-specific in any way. Perhaps by looking there, you can > Chris> find some inspiration? > > Here the immediate downside would be that stuff like auto-scaling does > not work out of the box. Which some people consider one of the selling > features of AWS, the prices for VM hosting being rather high. I think auto-scaling is orthogonal to this discussion. If we build a way to deploy an EC2 instance from a GuixSD operating system configuration file, then it implies that we also have the ability to create an AMI from the operating system configuration file. So auto-scaling is definitely possible. Whether or not auto-scaling works for a given use case depends on the use case, not on the mechanism by which the AMI used for auto-scaling was created. > Chris> I think it would be better to spend your energy on creating a > Chris> mechanism that allows an individual to build a GuixSD image > Chris> from their own operating system configuration file, import > Chris> that into EC2, and then launch an instance from it. If such > Chris> a feature were available in GuixSD, you could do it once from > Chris> a desktop/laptop with a slow internet connection to create a > Chris> "control server" in the cloud (with a fast internet > Chris> connection), and then you could run it from the control > Chris> server as needed to quickly spin up whatever other instances > Chris> you might need. > > I think the above steps could be shortened somewhat and automated, if > you know you're running on ec2. > > I don't see a way to cleanly import an image into AWS. This is however > different for OpenStack, there you have an image service that does just > what we need. I haven't tried it myself, but I suspect that the "VM Import" feature is the right approach. I think it's probably the best thing to try first. We would need to implement the mechanism for building a GuixSD system in whatever VM format EC2 requires, and we would need to implement a Guile interface for EC2 (or at least wrap the AWS CLI). > I'll try my hand at optimizing these steps on the weekend. Good luck! If you need anything, I'm happy to talk. Finally, I'd like to mention that although I work at AWS, this email correspondence is entirely my own personal opinion and does not reflect the views or opinions of my employer. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAljsgAIACgkQ3UCaFdgi Rp2FqhAAiPvVmxyoBjS2lN4sibTDse64Ac5H+96Ag/iyqigfMdHhQs50W5kVK+3c 1l9QxJOUQR/01boifGB8l6oMwn0x7nendfQ8ayFwMBuiMoT8cHSMFXteTpn3uN2e Z3JToV5OhOzaKBfRb/ca6668ilQLYAcMzaX7h9citF2iNEB76gnksMofCjyHBzel JeuZQE8jRyWYtY36qYqJ1Rzq9dMji9ASpzrWJ8VLTTk/IjC83kwWu4vWWZFIJY0L GdeBiHvcRnF9LSy6uhS5dD/Lus30d5O67hj/5T80thUYA2uc806lO0lnQDS5Alel AmcLqGA3vrzYQiuaGBFtx73FXG1hCFtKh82V+IkEWBYony4ESxoLC3b5OUmm7Akc eM3enkrcnrr/Use7Zz8pkHp0Lf68H1hHsuy4MDWosM7oCzSCmDUJJpJaxsy1HPtm zRYIDBdP8rZeQR3R/N68RxTTGXc24PzAWJ5Ac/NL4GE7aDN7aCYUw4EWMAhkUKYJ /kd8dgxQcNV4oACgHKG6pNjKwmA6FpfTx8wnU/3TkJoor9t6ysvMzVTZhEt5uPpB P+EZUFe29ZcSFQOg+RlMO4XZle3ylu4RWQV/Z6grFSqbYqcyIfWVVD5V7g0WUGeX Sfn7APkOs5FQDXAaCgIsDkHjjd4WWuL7BOIT7JDTHYLAz5stt7s= =kT9r -----END PGP SIGNATURE----- --=-=-=--