* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-11-30 4:50 ` Christopher Allan Webber
@ 2016-11-30 15:08 ` Marius Bakke
2016-11-30 18:18 ` Christopher Allan Webber
2016-11-30 16:41 ` Chris Marusich
` (3 subsequent siblings)
4 siblings, 1 reply; 15+ messages in thread
From: Marius Bakke @ 2016-11-30 15:08 UTC (permalink / raw)
To: Christopher Allan Webber, help-guix
[-- Attachment #1: Type: text/plain, Size: 3633 bytes --]
Christopher Allan Webber <cwebber@dustycloud.org> writes:
> Christopher Allan Webber writes:
>
>> What am I doing wrong? I'm not totally sure... I feel like I'm
>> navigating a jungle out here in the OpenStack / Rackspace docs. Here's
>> one thing I found:
>>
>> https://community.rackspace.com/products/f/25/t/7186
>>
>> So I probably need to execute this "import" command. I guess that's
>> what's next...
>
> Well, I did a bit more exploration tonight with *some* progress, but
> still not success. I followed the above link and got a task to generate
> an image based off my file. Looks like the task took! And it showed up
> in "image-list" with the glance command line client. Sounds like
> progress!
>
> And hey, if I try to create the server now via the web UI... if I look
> to create an image based off the images list from
> "Saved -> Deleted Servers (!!!)" menu, I indeed see my image listed.
> Cool! So I select that and click "create server".
>
> Ok... I wait a bit. It says it's initializing it!
>
> Uhoh, suddenly the status turns to ERROR. What's ERROR? I don't know.
> It says ERROR, and it's red. Hovering over it suggests I ask support.
> Hm.
>
> I wonder if I used the Nova command line client if I'd get more
> information, or if there's a way to query the API to get more info.
>
> Still, that's *some* progress... I kicked off generating an image
> generated via GuixSD, even if it didn't work at all... :)
>
> Relatedly! User dvc in #guix on freenode suggests looking at
> https://www.vultr.com/ which looks quite affordable and hey! It has a
> "custom ISO" option. If we can convert our USB boot stick thingy
> (presumably via xorriso) we could try generating a base server image
> from there. I'd prefer to have a workflow where I go from handing off
> something made with "guix system vm-image" to some API, but maybe in the
> meanwhile Vultr would be a lower barrier to entry.
>
> In the meanwhile, anyone familiar enough with Nova or Rackspace want to
> give me hints on how to find out more about what ERROR means, more
> specifically? ;)
I've had the questionable privilege of working on Openstack for some
time, and indeed have a GuixSD system running on it. I can tell you that
any errors you see are not necessarily representative of what's
happening in the back-end, although it does sound like the Rackspace GUI
just spits out a generic message, instead of a flat-out lie..
The image I used was created by installing a bare-bones GuixSD to an LVM
device (e.g. with Qemu), then dumping this with `dd` before first boot
to create a "RAW" image. RAW is supported (required, actually) by Glance
if Ceph is used as the backing storage, but Rackspace only supports VHD.
I haven't looked into the fstab of the generated VM image, but it may be
hard-coded to '/dev/vda' whereas Xen creates '/dev/xvda' (IIRC). Though
it should at least try to boot if that was the case.
My best advice is to try doing a normal install in a VM from the USB
image (qcow2 probably works) and then convert it to the format your
cloud platform expects, instead of booting the VM image directly. I used
GPT with a "bios_grub" partition FWIW.
Make sure to use (title 'label) and appropriate FS labels, since the
root device path may vary between cloud platforms.
On a vaguely related note, I'm slowly working on a "native" GuixSD
hosting platform, where you will be able to submit (and share!)
configuration files and get a VM and/or disk image back. It's still a
long way off, but I could use some help building the web front-end once
the back-end is ready. Feel free to contact me for more details :)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-11-30 15:08 ` Marius Bakke
@ 2016-11-30 18:18 ` Christopher Allan Webber
0 siblings, 0 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-11-30 18:18 UTC (permalink / raw)
To: Marius Bakke; +Cc: help-guix
Marius Bakke writes:
> I've had the questionable privilege of working on Openstack for some
> time, and indeed have a GuixSD system running on it. I can tell you that
> any errors you see are not necessarily representative of what's
> happening in the back-end, although it does sound like the Rackspace GUI
> just spits out a generic message, instead of a flat-out lie..
Yeah it was pretty generic, so I have no idea what happened. :)
> The image I used was created by installing a bare-bones GuixSD to an LVM
> device (e.g. with Qemu), then dumping this with `dd` before first boot
> to create a "RAW" image. RAW is supported (required, actually) by Glance
> if Ceph is used as the backing storage, but Rackspace only supports VHD.
Interesting!
> I haven't looked into the fstab of the generated VM image, but it may be
> hard-coded to '/dev/vda' whereas Xen creates '/dev/xvda' (IIRC). Though
> it should at least try to boot if that was the case.
Yeah, my configuration was set up for /dev/sda I think. I guess I could
do a system reconfigure switching it out to /dev/vda or whatever at
least minute... I dunno.
> My best advice is to try doing a normal install in a VM from the USB
> image (qcow2 probably works) and then convert it to the format your
> cloud platform expects, instead of booting the VM image directly. I used
> GPT with a "bios_grub" partition FWIW.
Out of curiosity, why do you think this would be an improvement?
> Make sure to use (title 'label) and appropriate FS labels, since the
> root device path may vary between cloud platforms.
Right, though labels might vary between platforms too, correct?
> On a vaguely related note, I'm slowly working on a "native" GuixSD
> hosting platform, where you will be able to submit (and share!)
> configuration files and get a VM and/or disk image back. It's still a
> long way off, but I could use some help building the web front-end once
> the back-end is ready. Feel free to contact me for more details :)
Now this I'm really interested in! I'll ping you off-list to discuss
this more. :)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-11-30 4:50 ` Christopher Allan Webber
2016-11-30 15:08 ` Marius Bakke
@ 2016-11-30 16:41 ` Chris Marusich
2016-12-01 16:52 ` Christopher Allan Webber
[not found] ` <87r35t42qe.fsf@we.make.ritual.n0.is>
` (2 subsequent siblings)
4 siblings, 1 reply; 15+ messages in thread
From: Chris Marusich @ 2016-11-30 16:41 UTC (permalink / raw)
To: Christopher Allan Webber; +Cc: help-guix
[-- Attachment #1.1: Type: text/plain, Size: 2508 bytes --]
Hi,
Reading Chris' report, I was inspired to try the same thing on EC2. I
don't know much about Linode or Rackspace, but I do know EC2. I'm happy
to report that I was successfully in creating a GuixSD EC2 instance.
Here's what I did. Essentially, all I wanted to do was run "guix system
init" on the root device of an EC2 instance. I accomplished that by
running "guix system init" on a loopback device (on my laptop), copying
the file which backed the loopback device onto an EC2 instance, writing
the file to an EBS volume attached to the instance, and then replacing
the EC2 instance's root EBS volume with the newly written EBS volume.
Here are the steps I took, in detail. Hopefully it's clear enough for
someone else to try it out if they want to.
(From the AWS Management Console)
* Launch an instance. I used an EBS-backed Ubuntu AMI using HVM
virtualization in the us-west-2 region. This one: ami-cbd276ab. It
needs to be EBS-backed because I want to be able to detach and attach
the volumes easily; it needs to be HVM virtualization because that way
it will boot using the GRUB I will install in the GuixSD system's root
device (paravirtualized EC2 instances follow a different boot
process). In the web UI, before launching the instance, I removed the
ephemeral storage from it (since I didn't need it), and I added an
extra 30 GiB EBS volume (this will become the GuixSD system's root
device). I also gave the instance a name tag to keep track of it in
the AWS Management Console. I used an m3.medium type instance, since
that is the smallest type which would give me consistent performance.
Here's a quick link to launch the AMI I used:
https://console.aws.amazon.com/ec2/home?region=us-west-2#launchAmi=ami-cbd276ab
(On my local computer)
* Generate a hashed password for use in the operating system config (see
"(guile) Encryption" for details). Here, the password is
57HdiXRftjV55zpXh5k3 (don't worry, I've changed it :-)):
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (crypt "57HdiXRftjV55zpXh5k3" "$1$JoT5zkpN")
$2 = "$1$JoT5zkpN$1CEM/T/UZ5a4nP1Kh8QoZ/"
scheme@(guile-user)>
--8<---------------cut here---------------end--------------->8---
* Make the following changes to the "bare-bones" operating system
configuration file that comes with Guix
(gnu/system/examples/bare-bones.tmpl), in order to enable remote login
using that password:
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: patch --]
[-- Type: text/x-patch, Size: 1670 bytes --]
--- /home/marusich/guix/gnu/system/examples/bare-bones.tmpl 2016-04-02 13:35:30.226642261 -0700
+++ /home/marusich/bare-bones-with-ssh.scm 2016-11-30 04:59:23.447719406 -0800
@@ -10,11 +10,10 @@
(timezone "Europe/Berlin")
(locale "en_US.UTF-8")
- ;; Assuming /dev/sdX is the target hard disk, and "my-root" is
- ;; the label of the target root file system.
- (bootloader (grub-configuration (device "/dev/sdX")))
+ ;; Install GRUB to the loopback device so it goes into our disk.img.
+ (bootloader (grub-configuration (device "/dev/loop0")))
(file-systems (cons (file-system
- (device "my-root")
+ (device "root")
(title 'label)
(mount-point "/")
(type "ext4"))
@@ -25,8 +24,9 @@
;; empty password.
(users (cons (user-account
(name "alice")
- (comment "Bob's sister")
+ (comment "Down the rabbit hole we go")
(group "users")
+ (password "$1$JoT5zkpN$1CEM/T/UZ5a4nP1Kh8QoZ/")
;; Adding the account to the "wheel" group
;; makes it a sudoer. Adding it to "audio"
@@ -43,5 +43,7 @@
;; Add services to the baseline: a DHCP client and
;; an SSH server.
(services (cons* (dhcp-client-service)
- (lsh-service #:port-number 2222)
+ ;; Don't use lsh since it requires manual
+ ;; interaction at first boot.
+ (service openssh-service-type (openssh-configuration))
%base-services)))
[-- Attachment #1.3: Type: text/plain, Size: 3599 bytes --]
* Make a sparse file (to save space) to be our disk image:
dd if=/dev/zero of=disk.img bs=1 count=0 seek=30GiB
* Partition it (I suspect that toggling the boot flag is unnecessary,
but I did it just to be safe):
parted -s disk.img mklabel msdos
parted -s disk.img mkpart primary ext4 10M 100%
parted -s disk.img toggle 1 boot
parted -s disk.img print
* Set up a loopback device:
sudo losetup -P /dev/loop0 disk.img
* Make a file system:
sudo mkfs.ext4 -L root /dev/loop0p1
* Mount it:
sudo mount /dev/loop0p1 /mnt
* Init the system:
sudo guix system init ~/bare-bones-with-ssh.scm /mnt
* Grab some coffee.
* Unmount it:
sudo umount /mnt
* Detach the loopback device:
sudo losetup -d /dev/loop0
* Compress the disk image, so we can copy it to the instance faster.
gzip disk.img
* Copy the image to the instance:
scp -i ~/.ssh/marusich-ec2.pem disk.img.gz ubuntu@ec2-54-214-139-189.us-west-2.compute.amazonaws.com:/tmp
* Log into the instance (Ubuntu EC2 images always use the "ubuntu" user,
and the marusich-ec2.pem key is the one that I generated when
launching the instance using the AWS Management Console earlier).
ssh -i ~/.ssh/marusich-ec2.pem ubuntu@ec2-54-214-139-189.us-west-2.compute.amazonaws.com
(While logged into the instance)
* Write the disk image to the target (30 GiB) EBS volume (/dev/xvdf is
where it's attached):
sudo bash -c 'gzip -d -c /tmp/disk.img.gz > /dev/xvdf'
* Grab another cup of coffee. This takes a while.
(From the AWS Management Console)
* Stop the instance.
* Detach all its EBS volumes.
* Attach the target (30 GiB) volume to the instance as '/dev/sda1'.
This will actually attach the volume at '/dev/xvda'. I don't know why
EC2 requires us to attach it as '/dev/sda1', but it does; it's weird.
* Start the instance.
* Now ssh into your new EC2 instance running GuixSD!
ssh alice@ec2-54-189-100-63.us-west-2.compute.amazonaws.com
* From here, you can reconfigure to your heart's content. I've
confirmed that running "guix system reconfigure" works.
Some thoughts about the experience:
* The "get console screenshot" and "get system log" features (of Amazon
EC2) don't work with the GuixSD instance; not sure why, but it made
troubleshooting a little harder. Maybe tty settings need tweaking?
* It would have been nice if I could have done this without relying on
an existing EC2 instance. In the future, we could consider using the
EC2 APIs to import an image, similar to what is described here:
http://docs.aws.amazon.com/vm-import/latest/userguide/import-vm-image.html
* There was some discussion in IRC about whether or not it would make
sense to have a "base AMI" for GuixSD. Dave made a good case for
avoiding that practice [1], which goes against the spirit of GuixSD.
Instead, for using GuixSD with virtualization services like EC2, in
the long term, it seems preferable to make it easy to init the system
you want (including the ssh keys and all) and then import that system
into the EC2 service (or other virtualization service). That's
basically what I did here, albeit via a hacky manual method.
* There is currently no support for declaring a user's authorized_keys
for SSH. Just like we have a way to declare a user's password, it
would be great to have a way to declare a user's authorized_keys.
There was some discussion about this on IRC also [1], with some
promising paths forward.
[1] https://gnunet.org/bot/log/guix/2016-11-30
--
Chris
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-11-30 16:41 ` Chris Marusich
@ 2016-12-01 16:52 ` Christopher Allan Webber
0 siblings, 0 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-12-01 16:52 UTC (permalink / raw)
To: Chris Marusich; +Cc: help-guix
Chris Marusich writes:
> Hi,
>
> Reading Chris' report, I was inspired to try the same thing on EC2. I
> don't know much about Linode or Rackspace, but I do know EC2. I'm happy
> to report that I was successfully in creating a GuixSD EC2 instance.
>
> Here's what I did. Essentially, all I wanted to do was run "guix system
> init" on the root device of an EC2 instance. I accomplished that by
> running "guix system init" on a loopback device (on my laptop), copying
> the file which backed the loopback device onto an EC2 instance, writing
> the file to an EBS volume attached to the instance, and then replacing
> the EC2 instance's root EBS volume with the newly written EBS volume.
Wow, great! Thank you for trying it, and documenting it :)
^ permalink raw reply [flat|nested] 15+ messages in thread
[parent not found: <87r35t42qe.fsf@we.make.ritual.n0.is>]
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-11-30 4:50 ` Christopher Allan Webber
` (2 preceding siblings ...)
[not found] ` <87r35t42qe.fsf@we.make.ritual.n0.is>
@ 2016-12-02 4:06 ` Christopher Allan Webber
2016-12-02 6:22 ` Chris Marusich
2016-12-02 19:51 ` Christopher Baines
2016-12-02 20:53 ` Christopher Allan Webber
4 siblings, 2 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-12-02 4:06 UTC (permalink / raw)
To: help-guix
Christopher Allan Webber writes:
> Relatedly! User dvc in #guix on freenode suggests looking at
> https://www.vultr.com/ which looks quite affordable and hey! It has a
> "custom ISO" option. If we can convert our USB boot stick thingy
> (presumably via xorriso) we could try generating a base server image
> from there. I'd prefer to have a workflow where I go from handing off
> something made with "guix system vm-image" to some API, but maybe in the
> meanwhile Vultr would be a lower barrier to entry.
I decided to explore this a bit more today, figuring converting Guix's
USB image to .iso couldn't be that hard. Well, turns out it's another
rabbit hole, and I didn't reach the end of this one yet either! But I
did make some progress, and thought I'd document what I did. (Probably
the loopback step can be skipped if we get to making a proper Guix
command out of this using some gexp. But I decided starting with the
USB image was the path of least resistance.)
Let's start up a little environment:
$ guix environment --ad-hoc xorriso parted
So the key tool here is xorriso, a rather featureful program for ISO
9960 filesystems (read: data CD-ROMs). But before we use that, we need
to fetch the usb image (or generate it) and mount it. Assume you've got
the USB image handy (and uncompressed); then we want to mount it via a
loopback device, but we need to get the right offset to find out what
byte the root partition starts from. Let's find out where that is.
# We need to find out how many bytes the offset is
cwebber@oolong:/tmp$ parted guixsd-usb-install-0.11.0.x86_64-linux
# This switches it to bytes output
(parted) unit B
(parted) print
Model: (file)
Disk /tmp/guixsd-usb-install-0.11.0.x86_64-linux: 943718400B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1048576B 934281727B 933233152B primary ext4 boot
(parted) quit
Ok, so we can take that "start" number (be sure to strip the trailing
'B'):
$ sudo mount -o ro,loop,offset=1048576 guixsd-usb-install-0.11.0.x86_64-linux /mnt/tmp
Now we can write it to the .iso image, packaging up all those files that
appear in the usb image:
# flags: verbose, Rock Ridge filesystem info, output path, input directory
$ sudo xorrisofs -v -R -o ./guixsd-usb-install-0.11.0.x86_64-linux.iso /mnt/tmp/
GNU xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
Drive current: -outdev 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 242g free
xorriso : UPDATE : 27100 files added in 1 seconds
xorriso : UPDATE : 47400 files added in 2 seconds
xorriso : UPDATE : 50200 files added in 3 seconds
[.....]
Written to medium : 351969 sectors at LBA 0
Writing to 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso' completed successfully.
Except, oops! This won't boot. We need to put GRUB on it.
Well here's where I'm stuck for tonight. I don't know exactly what's
needed; maybe either the -b flag, or --grub2-mbr, or some combination.
The man page is a little bit overwhelming (I mean, xorrisofs is clearly
featureful, credit there):
https://www.gnu.org/software/xorriso/man_1_xorrisofs.html
But how do I generate the right GRUB stuff to put there? Can I pull it
off the USB image? Generate it separately?
This web page is very long but appears to have the appropriate info (and
unfortunately requires running arbitrary javascript to even render):
http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html
... so I guess the next steps are following roughly what's described at
the bottom of the page?
I wonder how other distros do this step.
The light at the end of this particular rabbit hole feels close, but not
there yet! As always, external input from those who are more
knowledgable: more than welcome.
- Chris
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-12-02 4:06 ` Christopher Allan Webber
@ 2016-12-02 6:22 ` Chris Marusich
2016-12-02 17:39 ` Christopher Allan Webber
2016-12-02 19:51 ` Christopher Baines
1 sibling, 1 reply; 15+ messages in thread
From: Chris Marusich @ 2016-12-02 6:22 UTC (permalink / raw)
To: Christopher Allan Webber; +Cc: help-guix
[-- Attachment #1: Type: text/plain, Size: 4592 bytes --]
Hi Chris,
Sounds like you've been having fun :-)
Christopher Allan Webber <cwebber@dustycloud.org> writes:
> Christopher Allan Webber writes:
>
>> Relatedly! User dvc in #guix on freenode suggests looking at
>> https://www.vultr.com/ which looks quite affordable and hey! It has a
>> "custom ISO" option. If we can convert our USB boot stick thingy
>> (presumably via xorriso) we could try generating a base server image
>> from there. I'd prefer to have a workflow where I go from handing off
>> something made with "guix system vm-image" to some API, but maybe in the
>> meanwhile Vultr would be a lower barrier to entry.
>
> I decided to explore this a bit more today, figuring converting Guix's
> USB image to .iso couldn't be that hard. Well, turns out it's another
> rabbit hole, and I didn't reach the end of this one yet either!
You can say that again! ISOs and bootable CDs are an area that is often
complicated, apparently can be done in multiple ways, and is often
misunderstood or glossed over in most Internet discussions.
> Assume you've got the USB image handy (and uncompressed); then we want
> to mount it via a loopback device, but we need to get the right offset
> to find out what byte the root partition starts from. Let's find out
> where that is.
>
> # We need to find out how many bytes the offset is
> cwebber@oolong:/tmp$ parted guixsd-usb-install-0.11.0.x86_64-linux
> # This switches it to bytes output
> (parted) unit B
> (parted) print
> Model: (file)
> Disk /tmp/guixsd-usb-install-0.11.0.x86_64-linux: 943718400B
> Sector size (logical/physical): 512B/512B
> Partition Table: msdos
> Disk Flags:
>
> Number Start End Size Type File system Flags
> 1 1048576B 934281727B 933233152B primary ext4 boot
>
> (parted) quit
>
> Ok, so we can take that "start" number (be sure to strip the trailing
> 'B'):
>
> $ sudo mount -o ro,loop,offset=1048576 guixsd-usb-install-0.11.0.x86_64-linux /mnt/tmp
FYI, you can also mount the file on a loopback device, like I did in my
EC2 example. If you use the -P option to losetup, it'll automatically
create a device file for the partition, which you can then mount. That
way, there is no need to find/specify the offset.
> Now we can write it to the .iso image, packaging up all those files that
> appear in the usb image:
>
> # flags: verbose, Rock Ridge filesystem info, output path, input directory
> $ sudo xorrisofs -v -R -o ./guixsd-usb-install-0.11.0.x86_64-linux.iso /mnt/tmp/
> GNU xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
>
> Drive current: -outdev 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso'
> Media current: stdio file, overwriteable
> Media status : is blank
> Media summary: 0 sessions, 0 data blocks, 0 data, 242g free
> xorriso : UPDATE : 27100 files added in 1 seconds
> xorriso : UPDATE : 47400 files added in 2 seconds
> xorriso : UPDATE : 50200 files added in 3 seconds
> [.....]
> Written to medium : 351969 sectors at LBA 0
> Writing to 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso' completed successfully.
>
> Except, oops! This won't boot. We need to put GRUB on it.
>
> Well here's where I'm stuck for tonight. I don't know exactly what's
> needed; maybe either the -b flag, or --grub2-mbr, or some combination.
> The man page is a little bit overwhelming (I mean, xorrisofs is clearly
> featureful, credit there):
>
> https://www.gnu.org/software/xorriso/man_1_xorrisofs.html
>
> But how do I generate the right GRUB stuff to put there? Can I pull it
> off the USB image? Generate it separately?
>
> This web page is very long but appears to have the appropriate info (and
> unfortunately requires running arbitrary javascript to even render):
>
> http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html
>
> ... so I guess the next steps are following roughly what's described at
> the bottom of the page?
That is actually the exact webpage I was going to link to you. What you
need to do (I think) is make the CD bootable, and the common way to do
that, apparently, is to do it via El Torito. I've never done it,
myself, but my understanding is that that is what needs to be done.
In my opinion, unless the virtualization service provides no other easy
way to import an image for booting, I doubt that making a bootable ISO
is the simplest solution. But it's good to experiment!
--
Chris
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-12-02 6:22 ` Chris Marusich
@ 2016-12-02 17:39 ` Christopher Allan Webber
0 siblings, 0 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-12-02 17:39 UTC (permalink / raw)
To: Chris Marusich; +Cc: help-guix
Chris Marusich writes:
> Hi Chris,
>
> Sounds like you've been having fun :-)
Yes, though I'd rather have things working than have fun right now ;)
> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>
>> Christopher Allan Webber writes:
>>
>>> Relatedly! User dvc in #guix on freenode suggests looking at
>>> https://www.vultr.com/ which looks quite affordable and hey! It has a
>>> "custom ISO" option. If we can convert our USB boot stick thingy
>>> (presumably via xorriso) we could try generating a base server image
>>> from there. I'd prefer to have a workflow where I go from handing off
>>> something made with "guix system vm-image" to some API, but maybe in the
>>> meanwhile Vultr would be a lower barrier to entry.
>>
>> I decided to explore this a bit more today, figuring converting Guix's
>> USB image to .iso couldn't be that hard. Well, turns out it's another
>> rabbit hole, and I didn't reach the end of this one yet either!
>
> You can say that again! ISOs and bootable CDs are an area that is often
> complicated, apparently can be done in multiple ways, and is often
> misunderstood or glossed over in most Internet discussions.
>
>> Assume you've got the USB image handy (and uncompressed); then we want
>> to mount it via a loopback device, but we need to get the right offset
>> to find out what byte the root partition starts from. Let's find out
>> where that is.
>>
>> # We need to find out how many bytes the offset is
>> cwebber@oolong:/tmp$ parted guixsd-usb-install-0.11.0.x86_64-linux
>> # This switches it to bytes output
>> (parted) unit B
>> (parted) print
>> Model: (file)
>> Disk /tmp/guixsd-usb-install-0.11.0.x86_64-linux: 943718400B
>> Sector size (logical/physical): 512B/512B
>> Partition Table: msdos
>> Disk Flags:
>>
>> Number Start End Size Type File system Flags
>> 1 1048576B 934281727B 933233152B primary ext4 boot
>>
>> (parted) quit
>>
>> Ok, so we can take that "start" number (be sure to strip the trailing
>> 'B'):
>>
>> $ sudo mount -o ro,loop,offset=1048576 guixsd-usb-install-0.11.0.x86_64-linux /mnt/tmp
>
> FYI, you can also mount the file on a loopback device, like I did in my
> EC2 example. If you use the -P option to losetup, it'll automatically
> create a device file for the partition, which you can then mount. That
> way, there is no need to find/specify the offset.
Oh, that's useful... I didn't realize that :)
>> Now we can write it to the .iso image, packaging up all those files that
>> appear in the usb image:
>>
>> # flags: verbose, Rock Ridge filesystem info, output path, input directory
>> $ sudo xorrisofs -v -R -o ./guixsd-usb-install-0.11.0.x86_64-linux.iso /mnt/tmp/
>> GNU xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
>>
>> Drive current: -outdev 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso'
>> Media current: stdio file, overwriteable
>> Media status : is blank
>> Media summary: 0 sessions, 0 data blocks, 0 data, 242g free
>> xorriso : UPDATE : 27100 files added in 1 seconds
>> xorriso : UPDATE : 47400 files added in 2 seconds
>> xorriso : UPDATE : 50200 files added in 3 seconds
>> [.....]
>> Written to medium : 351969 sectors at LBA 0
>> Writing to 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso' completed successfully.
>>
>> Except, oops! This won't boot. We need to put GRUB on it.
>>
>> Well here's where I'm stuck for tonight. I don't know exactly what's
>> needed; maybe either the -b flag, or --grub2-mbr, or some combination.
>> The man page is a little bit overwhelming (I mean, xorrisofs is clearly
>> featureful, credit there):
>>
>> https://www.gnu.org/software/xorriso/man_1_xorrisofs.html
>>
>> But how do I generate the right GRUB stuff to put there? Can I pull it
>> off the USB image? Generate it separately?
>>
>> This web page is very long but appears to have the appropriate info (and
>> unfortunately requires running arbitrary javascript to even render):
>>
>> http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html
>>
>> ... so I guess the next steps are following roughly what's described at
>> the bottom of the page?
>
> That is actually the exact webpage I was going to link to you. What you
> need to do (I think) is make the CD bootable, and the common way to do
> that, apparently, is to do it via El Torito. I've never done it,
> myself, but my understanding is that that is what needs to be done.
Yeah I think we need to generate the Grub configuration, then pass to
Guix somehow. I don't totally understand how that is all done yet.
> In my opinion, unless the virtualization service provides no other easy
> way to import an image for booting, I doubt that making a bootable ISO
> is the simplest solution. But it's good to experiment!
Maybe not. I felt like it might be the easiest path after feeling like
the error message I got from the Rackspace image upload I made was
fairly opaque. Something where I could watch the output of something
from something resembling a console seemed appealing.
I should probably try to replicate the work you've done on AWS hosting
as a next step. I'm not wild about AWS, but at this point, maybe having
something working is most important.
I'll re-read what you posted anyway. Clearly there was some useful info
I missed!
- CHris
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-12-02 4:06 ` Christopher Allan Webber
2016-12-02 6:22 ` Chris Marusich
@ 2016-12-02 19:51 ` Christopher Baines
2016-12-02 23:20 ` Tobias Geerinckx-Rice
2016-12-03 18:24 ` Christopher Allan Webber
1 sibling, 2 replies; 15+ messages in thread
From: Christopher Baines @ 2016-12-02 19:51 UTC (permalink / raw)
To: help-guix
On 02/12/16 04:06, Christopher Allan Webber wrote:
> Except, oops! This won't boot. We need to put GRUB on it.
>
> Well here's where I'm stuck for tonight. I don't know exactly what's
> needed; maybe either the -b flag, or --grub2-mbr, or some combination.
> The man page is a little bit overwhelming (I mean, xorrisofs is clearly
> featureful, credit there):
>
> https://www.gnu.org/software/xorriso/man_1_xorrisofs.html
>
> But how do I generate the right GRUB stuff to put there? Can I pull it
> off the USB image? Generate it separately?
>
> This web page is very long but appears to have the appropriate info (and
> unfortunately requires running arbitrary javascript to even render):
>
> http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html
>
> ... so I guess the next steps are following roughly what's described at
> the bottom of the page?
>
> I wonder how other distros do this step.
Thanks for looking in to this Chris! I'm using Bytemark for personal
servers, and have tried and failed to install Guix from a Debian live
cd. But, they do support installing from arbitrary ISO images, this
would be great to get working.
The bash script which I guess Nix uses looks quite short [1], you might
find some useful inspiration there.
1:
https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.sh
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-12-02 19:51 ` Christopher Baines
@ 2016-12-02 23:20 ` Tobias Geerinckx-Rice
2016-12-03 19:13 ` Christopher Allan Webber
2016-12-03 18:24 ` Christopher Allan Webber
1 sibling, 1 reply; 15+ messages in thread
From: Tobias Geerinckx-Rice @ 2016-12-02 23:20 UTC (permalink / raw)
To: mail, help-guix
[-- Attachment #1.1: Type: text/plain, Size: 1842 bytes --]
Chris[es],
[If ‘commodity’ means ‘whatever this OpenStack thing might be’, or
anything involving comparison of pets to cattle, this is not that. But
this caught my eye:]
On 02/12/16 20:51, Christopher Baines wrote:
> Thanks for looking in to this Chris! I'm using Bytemark for personal
> servers, and have tried and failed to install Guix from a Debian
> live cd.
Interesting. That should work.
I've been running a pure (from scratch) GuixSD VPS[0] for the better
half of a year. I bootstrapped from a provider-supplied SystemRescueCD
image, since it has all required packages pre-installed, but any sane
live system should do.
Partition and format the target device. Mount that. Upload
shiny-new-system.scm. Get your swapon. Bind mount /gnu, /var/guix and
/tmp from temporary directories on /mnt, to prevent ‘guix system init
/mnt’ filling up the RAM disc. Watch the builds scroll by. Reboot. Done.
If I forgot anything, it can't have been much.
A similar approach should work on all hosting providers that support
booting live CDs that can run out of RAM. I suspect one could ‘snapshot’
the base system on providers that support it, to spin up additional
instances faster. I haven't tried; installing from scratch is just as
easy and keeps the bits fresh.
No idea how this relates to ‘commodity hosting’: SystemRescueCD happens
to require console access to set the root password for SSH, but after
that the entire deployment can be scripted. Mine are.
It's as far from underdocumented (to the point of being proprietary)
image layouts and shiny ‘API’s as possible, which is a nice bonus. :-)
Kind regards,
T G-R
[0]: <https://liteserver.nl/>: I happily endorse them, and can't wait to
send them a ‘Please add GuixSD-1.0.iso’ ticket when the time comes.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 476 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-12-02 23:20 ` Tobias Geerinckx-Rice
@ 2016-12-03 19:13 ` Christopher Allan Webber
0 siblings, 0 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-12-03 19:13 UTC (permalink / raw)
To: Tobias Geerinckx-Rice; +Cc: help-guix
Tobias Geerinckx-Rice writes:
> Chris[es],
>
> [If ‘commodity’ means ‘whatever this OpenStack thing might be’, or
> anything involving comparison of pets to cattle, this is not that. But
> this caught my eye:]
>
> On 02/12/16 20:51, Christopher Baines wrote:
>> Thanks for looking in to this Chris! I'm using Bytemark for personal
>> servers, and have tried and failed to install Guix from a Debian
>> live cd.
>
> Interesting. That should work.
>
> I've been running a pure (from scratch) GuixSD VPS[0] for the better
> half of a year. I bootstrapped from a provider-supplied SystemRescueCD
> image, since it has all required packages pre-installed, but any sane
> live system should do.
Hey great! And yeah, at this point, maybe if I'm looking at booting
from a .ISO file, while generating a .ISO file is a noble goal still,
I problably should just try installing from another distro.
> Partition and format the target device. Mount that. Upload
> shiny-new-system.scm. Get your swapon. Bind mount /gnu, /var/guix and
> /tmp from temporary directories on /mnt, to prevent ‘guix system init
> /mnt’ filling up the RAM disc. Watch the builds scroll by. Reboot. Done.
>
> If I forgot anything, it can't have been much.
>
> A similar approach should work on all hosting providers that support
> booting live CDs that can run out of RAM. I suspect one could ‘snapshot’
> the base system on providers that support it, to spin up additional
> instances faster. I haven't tried; installing from scratch is just as
> easy and keeps the bits fresh.
>
> No idea how this relates to ‘commodity hosting’: SystemRescueCD happens
> to require console access to set the root password for SSH, but after
> that the entire deployment can be scripted. Mine are.
>
> It's as far from underdocumented (to the point of being proprietary)
> image layouts and shiny ‘API’s as possible, which is a nice bonus. :-)
>
> Kind regards,
>
> T G-R
Sounds good :)
> [0]: <https://liteserver.nl/>: I happily endorse them, and can't wait to
> send them a ‘Please add GuixSD-1.0.iso’ ticket when the time comes.
Wow cool, some of those are quite cheap!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-12-02 19:51 ` Christopher Baines
2016-12-02 23:20 ` Tobias Geerinckx-Rice
@ 2016-12-03 18:24 ` Christopher Allan Webber
1 sibling, 0 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-12-03 18:24 UTC (permalink / raw)
To: Christopher Baines; +Cc: help-guix
Christopher Baines writes:
> On 02/12/16 04:06, Christopher Allan Webber wrote:
>> Except, oops! This won't boot. We need to put GRUB on it.
>>
>> Well here's where I'm stuck for tonight. I don't know exactly what's
>> needed; maybe either the -b flag, or --grub2-mbr, or some combination.
>> The man page is a little bit overwhelming (I mean, xorrisofs is clearly
>> featureful, credit there):
>>
>> https://www.gnu.org/software/xorriso/man_1_xorrisofs.html
>>
>> But how do I generate the right GRUB stuff to put there? Can I pull it
>> off the USB image? Generate it separately?
>>
>> This web page is very long but appears to have the appropriate info (and
>> unfortunately requires running arbitrary javascript to even render):
>>
>> http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html
>>
>> ... so I guess the next steps are following roughly what's described at
>> the bottom of the page?
>>
>> I wonder how other distros do this step.
>
> Thanks for looking in to this Chris! I'm using Bytemark for personal
> servers, and have tried and failed to install Guix from a Debian live
> cd. But, they do support installing from arbitrary ISO images, this
> would be great to get working.
>
> The bash script which I guess Nix uses looks quite short [1], you might
> find some useful inspiration there.
>
> 1:
> https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.sh
That looks VERY helpful, thank you!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hosting a GuixSD server on commodity hosting platforms, a journey
2016-11-30 4:50 ` Christopher Allan Webber
` (3 preceding siblings ...)
2016-12-02 4:06 ` Christopher Allan Webber
@ 2016-12-02 20:53 ` Christopher Allan Webber
4 siblings, 0 replies; 15+ messages in thread
From: Christopher Allan Webber @ 2016-12-02 20:53 UTC (permalink / raw)
To: help-guix
Hello everyone! So I got a very nice and useful response about getting
GuixSD working on Rackspace Cloud. I've gotten permission to post the
conversation verbatim, so I've done so below. But here's the tl;dr:
- In order for our image to work, we need to get XenServer and
nova-agent working and running on it. DHCP isn't used.
- Apparently the right format is "VHD in an OVF container"
but is hard to get right. I'm assuming we probably won't get
it right by guessing based on that description.
- So we probably should use this tool to do our first attempt
(then figure out from there how to generate such an image
ourselves without help):
http://rackerlabs.github.io/boot.rackspace.com/
- In order to use that, we'll need to supply a .iso, so I guess
in order to move forward with the Rackspace route, we'll need
to finish figuring out how to generate a .iso here too!
Thanks again to AlexOughton for the useful feedback.
> [Fri Dec 2 2016]
> <AlexOughton> hi, i just saw your question about GuixSD on #rackspace. you'll
> need to get both the XenServer tools and the nova-agent working,
> as Rackspace Cloud doesn't use DHCP [14:52]
> <paroneayea> hi! Ah I see :)
> <paroneayea> I've been documenting my attempt at getting GuixSD on Rackspace
> here:
> http://lists.gnu.org/archive/html/help-guix/2016-11/msg00073.html
> http://lists.gnu.org/archive/html/help-guix/2016-11/msg00074.html
> [14:54]
> <paroneayea> mind if I paste your reply here into the #guix irc channel and/or
> to the mailing list?
> <AlexOughton> sure. i don [14:55]
> <AlexOughton> sure. i don't know how far you're going to get with image import
> methods (as this can be flaky, and also requires VERY specific
> format). you might get better luck bootstrapping your own custom
> image this way: http://rackerlabs.github.io/boot.rackspace.com/
> [14:56]
> <AlexOughton> that only provides a few distros, but there are ways of pointing
> iPXE to your own ISO file to then boot that and perform an
> installation [14:57]
> <paroneayea> ok, thanks. Yeah I was having trouble finding any information
> about what the expected format is [15:00]
> <paroneayea> and there doesn't seem to be much info
> <AlexOughton> i believe it's VHD in an OVF container. [15:02]
> <AlexOughton> i think your success or failure is going to be down to whether
> or not the OS works on XenServer (which is what our hypervisors
> are), and whether or not the XenServer tools work. what kernel
> does guixsd use?
> <paroneayea> linux-libre 4.8.10 is latest I think [15:03]
> <paroneayea> you can install other kernels
> <paroneayea> but that's the default
> <AlexOughton> ok, well since it's a Linux my guess would be that it's possible
> to get XenServer tools working. and then the nova-agent is just
> python, so hopefully that's possible too [15:04]
> <paroneayea> cool, ok. I guess we need both nova-agent and XenServer then :)
> <AlexOughton> right. the boot.rackspace link i gave you above has
> documentation for getting both of those working as well
> <paroneayea> cool... well, I was just looking into making a .iso of the boot
> install media yesterday (currently we mostly use a USB install
> stick) so I guess there's extra motivation to do that now :)
> [15:05]
> <paroneayea> thanks for your help, this was a very useful conversation. Okay
> for me to just paste this conversation verbatim to the mailing
> list? [15:06]
> <AlexOughton> yep, no problem. as long as it's understood that i'm just
> answering as an individual, and that this isn't any kind of
> official rackspace support answer. because it's definitely not
> something which rackspace would be able to support for you
> [15:07]
> <paroneayea> understood :)
> <paroneayea> thank you!
> <AlexOughton> you're welcome
> <paroneayea> I really appreciate it :)
^ permalink raw reply [flat|nested] 15+ messages in thread