From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Re: Hosting a GuixSD server on commodity hosting platforms, a journey Date: Wed, 30 Nov 2016 08:41:17 -0800 Message-ID: <87a8cg7weq.fsf@gmail.com> References: <878ts212ic.fsf@dustycloud.org> <871sxt1sho.fsf@dustycloud.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:35999) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cC7wj-0007DD-Lz for help-guix@gnu.org; Wed, 30 Nov 2016 11:41:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cC7wi-0008U4-6T for help-guix@gnu.org; Wed, 30 Nov 2016 11:41:25 -0500 Received: from mail-pg0-x236.google.com ([2607:f8b0:400e:c05::236]:34166) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cC7wh-0008Td-R5 for help-guix@gnu.org; Wed, 30 Nov 2016 11:41:24 -0500 Received: by mail-pg0-x236.google.com with SMTP id x23so83849155pgx.1 for ; Wed, 30 Nov 2016 08:41:23 -0800 (PST) In-Reply-To: <871sxt1sho.fsf@dustycloud.org> (Christopher Allan Webber's message of "Tue, 29 Nov 2016 22:50:11 -0600") List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+gcggh-help-guix=m.gmane.org@gnu.org Sender: "Help-Guix" To: Christopher Allan Webber Cc: help-guix --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi, Reading Chris' report, I was inspired to try the same thing on EC2. I don't know much about Linode or Rackspace, but I do know EC2. I'm happy to report that I was successfully in creating a GuixSD EC2 instance. Here's what I did. Essentially, all I wanted to do was run "guix system init" on the root device of an EC2 instance. I accomplished that by running "guix system init" on a loopback device (on my laptop), copying the file which backed the loopback device onto an EC2 instance, writing the file to an EBS volume attached to the instance, and then replacing the EC2 instance's root EBS volume with the newly written EBS volume. Here are the steps I took, in detail. Hopefully it's clear enough for someone else to try it out if they want to. (From the AWS Management Console) * Launch an instance. I used an EBS-backed Ubuntu AMI using HVM virtualization in the us-west-2 region. This one: ami-cbd276ab. It needs to be EBS-backed because I want to be able to detach and attach the volumes easily; it needs to be HVM virtualization because that way it will boot using the GRUB I will install in the GuixSD system's root device (paravirtualized EC2 instances follow a different boot process). In the web UI, before launching the instance, I removed the ephemeral storage from it (since I didn't need it), and I added an extra 30 GiB EBS volume (this will become the GuixSD system's root device). I also gave the instance a name tag to keep track of it in the AWS Management Console. I used an m3.medium type instance, since that is the smallest type which would give me consistent performance. Here's a quick link to launch the AMI I used: https://console.aws.amazon.com/ec2/home?region=3Dus-west-2#launchAmi=3Dam= i-cbd276ab (On my local computer) * Generate a hashed password for use in the operating system config (see "(guile) Encryption" for details). Here, the password is 57HdiXRftjV55zpXh5k3 (don't worry, I've changed it :-)): =2D-8<---------------cut here---------------start------------->8--- scheme@(guile-user)> (crypt "57HdiXRftjV55zpXh5k3" "$1$JoT5zkpN") $2 =3D "$1$JoT5zkpN$1CEM/T/UZ5a4nP1Kh8QoZ/" scheme@(guile-user)>=20 =2D-8<---------------cut here---------------end--------------->8--- * Make the following changes to the "bare-bones" operating system configuration file that comes with Guix (gnu/system/examples/bare-bones.tmpl), in order to enable remote login using that password: --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=patch Content-Transfer-Encoding: quoted-printable =2D-- /home/marusich/guix/gnu/system/examples/bare-bones.tmpl 2016-04-02 13= :35:30.226642261 -0700 +++ /home/marusich/bare-bones-with-ssh.scm 2016-11-30 04:59:23.447719406 -0= 800 @@ -10,11 +10,10 @@ (timezone "Europe/Berlin") (locale "en_US.UTF-8") =20 =2D ;; Assuming /dev/sdX is the target hard disk, and "my-root" is =2D ;; the label of the target root file system. =2D (bootloader (grub-configuration (device "/dev/sdX"))) + ;; Install GRUB to the loopback device so it goes into our disk.img. + (bootloader (grub-configuration (device "/dev/loop0"))) (file-systems (cons (file-system =2D (device "my-root") + (device "root") (title 'label) (mount-point "/") (type "ext4")) @@ -25,8 +24,9 @@ ;; empty password. (users (cons (user-account (name "alice") =2D (comment "Bob's sister") + (comment "Down the rabbit hole we go") (group "users") + (password "$1$JoT5zkpN$1CEM/T/UZ5a4nP1Kh8QoZ/") =20 ;; Adding the account to the "wheel" group ;; makes it a sudoer. Adding it to "audio" @@ -43,5 +43,7 @@ ;; Add services to the baseline: a DHCP client and ;; an SSH server. (services (cons* (dhcp-client-service) =2D (lsh-service #:port-number 2222) + ;; Don't use lsh since it requires manual + ;; interaction at first boot. + (service openssh-service-type (openssh-configuration)) %base-services))) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * Make a sparse file (to save space) to be our disk image: dd if=3D/dev/zero of=3Ddisk.img bs=3D1 count=3D0 seek=3D30GiB * Partition it (I suspect that toggling the boot flag is unnecessary, but I did it just to be safe): parted -s disk.img mklabel msdos parted -s disk.img mkpart primary ext4 10M 100% parted -s disk.img toggle 1 boot parted -s disk.img print * Set up a loopback device: sudo losetup -P /dev/loop0 disk.img * Make a file system: sudo mkfs.ext4 -L root /dev/loop0p1 * Mount it: sudo mount /dev/loop0p1 /mnt * Init the system: sudo guix system init ~/bare-bones-with-ssh.scm /mnt * Grab some coffee. * Unmount it: sudo umount /mnt * Detach the loopback device: sudo losetup -d /dev/loop0 * Compress the disk image, so we can copy it to the instance faster. gzip disk.img * Copy the image to the instance: scp -i ~/.ssh/marusich-ec2.pem disk.img.gz ubuntu@ec2-54-214-139-189.us-wes= t-2.compute.amazonaws.com:/tmp * Log into the instance (Ubuntu EC2 images always use the "ubuntu" user, and the marusich-ec2.pem key is the one that I generated when launching the instance using the AWS Management Console earlier). ssh -i ~/.ssh/marusich-ec2.pem ubuntu@ec2-54-214-139-189.us-west-2.compute.= amazonaws.com (While logged into the instance) * Write the disk image to the target (30 GiB) EBS volume (/dev/xvdf is where it's attached): sudo bash -c 'gzip -d -c /tmp/disk.img.gz > /dev/xvdf' * Grab another cup of coffee. This takes a while. (From the AWS Management Console) * Stop the instance. * Detach all its EBS volumes. * Attach the target (30 GiB) volume to the instance as '/dev/sda1'. This will actually attach the volume at '/dev/xvda'. I don't know why EC2 requires us to attach it as '/dev/sda1', but it does; it's weird. * Start the instance. * Now ssh into your new EC2 instance running GuixSD! ssh alice@ec2-54-189-100-63.us-west-2.compute.amazonaws.com * From here, you can reconfigure to your heart's content. I've confirmed that running "guix system reconfigure" works. Some thoughts about the experience: * The "get console screenshot" and "get system log" features (of Amazon EC2) don't work with the GuixSD instance; not sure why, but it made troubleshooting a little harder. Maybe tty settings need tweaking? * It would have been nice if I could have done this without relying on an existing EC2 instance. In the future, we could consider using the EC2 APIs to import an image, similar to what is described here: http://docs.aws.amazon.com/vm-import/latest/userguide/import-vm-image.html * There was some discussion in IRC about whether or not it would make sense to have a "base AMI" for GuixSD. Dave made a good case for avoiding that practice [1], which goes against the spirit of GuixSD. Instead, for using GuixSD with virtualization services like EC2, in the long term, it seems preferable to make it easy to init the system you want (including the ssh keys and all) and then import that system into the EC2 service (or other virtualization service). That's basically what I did here, albeit via a hacky manual method. * There is currently no support for declaring a user's authorized_keys for SSH. Just like we have a way to declare a user's password, it would be great to have a way to declare a user's authorized_keys. There was some discussion about this on IRC also [1], with some promising paths forward. [1] https://gnunet.org/bot/log/guix/2016-11-30 =2D-=20 Chris --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYPwEtAAoJEN1AmhXYIkadVRUQANV3E083F2CA3aiRR++/uGCs vAPw36LL9ncV5fiI8QNO7LPL5fvKsLxTbX23he8XPwna0ZKgvK3W41dpKDS5t8MO Wn+Kmnh4Ss/9emp55SCPFF2xoVtMT4NNv75PnAEw472rVNNsfw93cbNLe5ZTaNcZ 7m0EX5lJQhqzZ1iHQe4yVfIWPkUaKyRQEi9jd77tXY7oF9ssS8T+yTR6xw3fGl2F T8O4uCodUNyI1hcNoIAHQZBct17FuPgt/vSzyFkqUsI18Y8/Gs4z1frU0ZkqLq7b l3tq4MPhkvETt4KYMBMP0b+B6pk5uP3rJc2Ryt+1EJGET7W8EUwhfxsYSiE1zyWJ 2IyddZxDeRUbYdUL1g+BVmkuemZI6phyxmeKgvlTnkHB5+SkQ2V1HzWq2/pXXzcV fnqnnEgjdczzXTI5Km4U2HZF++zUhElMLU19Ey0vwuowtj7MYuhWt8In34/D+Rvf 8hHV1MHFM8+BImLw3+h9VnXIGQf8+M1kujdOoDFdMbSzRPjxiHI4gwq3BY/3PZ0I QJPVRK2hDWfadv3iSHH6O6FbFN+hrR2s1OlxTrtreb1BxpplbJExaJC3PJ2TVFup XOJA3COAyQjwqfEI/So0yj19BtLCmwJukF3y4ozEq0ra+HzzwG3q+kvPr1mbKWzt NK0lB0cHudS/gBJUUXSn =qtaj -----END PGP SIGNATURE----- --==-=-=--