Guix System on a GCE VM -- Success

Guix System on a GCE VM -- Success

[elaexuotee]

[-- Attachment #1.1: Type: text/plain, Size: 3176 bytes --]

Attached is a minimal system configuration I have used to successfully
create a virtual machine instance on Google's Compute Engine (GCE).

The stock QEMU image [1] fails to boot under GCE, and since there is some past
interest [0], I figure it may be helpful to share the salient issues and how I
deal with them in the config.scm.

The boot problem fix boils down to this:

    (initrd-modules (cons "virtio_scsi" %base-initrd-modules))

Referencing Google's documentation on operating system requirements [2], we see
that their VMs run with Virtio-SCSI controllers; however, it just so happens
that the corresponding kernel module, virtio_scsi, is not included in the
initrd by default. Would it make sense to change this default?

That should be enough to get a working GCE instance running, provided you
follow Google's requirements for importing custom images [3]. It's not
challenging, but it takes time to familiarize oneself with GCE and the
associated toolchain.

To simplify the process I created a script capable of setting up a GCE instance
from a local qcow2 image, which I am also attaching. The script is intentially
not a simple turnkey solution, since you should be somewhat familiar with the
GCE infrastructure before using it. However, feel free to ask questions if
anything is unclear.

Returning back to config.scm, the only other non-obvious pieces deal with the
bootloader setup. On GCE at boot time, our only choice of interface is a serial
console, so the following set that up in case it becomes necessary:


      (bootloader (bootloader-configuration
                    ...
                    (serial-speed 115200)
                    (terminal-outputs '(serial))
                    (terminal-inputs '(serial))))

      (kernel-arguments '("console=ttyS0,115200"))

We set the baud rate to 115200 instead of the default 9600, for two reasons.
One it is essential that grub.cfg get populated with the following:

    serial [options]
    terminal_output serial
    terminal_input serial

and for some reason, bootloader-configuration seems to elide these lines
without serial-speed set to something other than 9600. Secondly, baud 115200
ends up providing a better shell experience anyway.

Finally, the attached config.scm sets a bootloader timeout of 0:

      (bootloader (bootloader-configuration
                    ...
                    (timeout 0)))

Since the vm is completely headless during normal bootup, anything else just
slows down boot times.


Finally, GCE allows community supported images [4]. If there is enough
interest, it might make sense to invest the few cents per month to maintain a
Guix System one. Thoughts?

Hopefully, the above is clear. I am happy to answer any questions.


[0]:https://lists.gnu.org/archive/html/guix-devel/2017-01/msg01815.html
[1]:https://ftp.gnu.org/gnu/guix/guix-system-vm-image-1.0.1.x86_64-linux.xz
[2]:https://cloud.google.com/compute/docs/images/building-custom-os
[3]:https://cloud.google.com/compute/docs/import/import-existing-image
[4]:https://cloud.google.com/compute/docs/images#community_supported_images


[-- Attachment #1.2: config.scm --]
[-- Type: text/plain, Size: 1158 bytes --]

(use-modules (gnu))
(use-service-modules networking ssh)

(operating-system
  (host-name "guixsd")
  (timezone "Europe/Berlin")
  (locale "en_US.utf8")

  (bootloader (bootloader-configuration
                (bootloader grub-bootloader)
                (target "/dev/sda")
                (theme (grub-theme))
                (timeout 0)
                (serial-speed 115200)
                (terminal-outputs '(serial))
                (terminal-inputs '(serial))))

  (kernel-arguments '("console=ttyS0,115200"))
  (initrd-modules (cons "virtio_scsi" %base-initrd-modules))

  (file-systems (cons (file-system
                        (device (uuid "b6a32092-b004-43ba-bab7-e4c8d5480026"))
                        (mount-point "/")
                        (type "ext4"))
                      %base-file-systems))

  (users (cons (user-account
                 (name "guest")
                 (group "users")
                 (supplementary-groups '("wheel")))
               %base-user-accounts))

  (services (append (list (service dhcp-client-service-type)
                          (service openssh-service-type))
                    %base-services)))

[-- Attachment #1.3: gce.sh --]
[-- Type: text/plain, Size: 10080 bytes --]

#!/usr/bin/env sh
set -o errexit -o noclobber -o nounset

## GCE Custom Image
#
# This script intends to be "executable documention" for the process of setting
# up a Google Compute Engine (GCE) virtual machine from a custom image.


usage() {
	cat <<-USAGE
	Usage: $(basename "${0}") [options] <command> [<command> [..]]

	This script aids in setting up Google Compute Engine with an instance
	running a custom image.

	Execute commands in order provided at the command line. The procedure
	for a fresh setup follows the order listed below in Status Commands.

	Setup Commands:
	  mkconf          Configure files on raw image partition
	  mkraw           Convert image to raw format
	  mksize          Grow image to integer multiple of GBs
	  mktar           Archive raw image
	  mkgsb           Create GS bucket

	  togsa           Upload archive to GS bucket
	  togci           Create GCE image from GS archive
	  togvm           Create GCE instance from GCE image

	Status Commands:
	  ckconf          Check contents of qcow2 image
	  ckraw           Check integrity of raw image
	  cksize          Check size of raw image
	  cktar           Check contents of tar archive
	  ckgsb           Check existence of GS bucket
	  ckgsa           Check existence of tar archive in GS bucket
	  ckgci           Check existence of GCE image
	  ckgvm           Check status of GCE instance

	  ttysro          View GCE instance's serial output
	  ttysrw          Connect to instance serial port

	Removal Commands:
	  rmgvm           Remove GCE instance
	  rmgci           Remove GCE image
	  rmgsa           Remove archive from GS bucket
	  rmgsb           Remove GS bucket

	Metainfo Commands:
	  help            Show this help message
	  vars            Print value of configuration variables

	Options:
	  -f <family>     Family name for GCE image
	  -v <version>    Version string of GCE image
	  -n <name>       Name of GCE image

	  -N <name>       Name of GCE instance
	  -s <subnet>     Subnet name of GCE instance
	  -m <type>       Machine type of GCE instance

	  -B <uri>        URI of GS bucket
	  -A <uri>        URI of GS tar file

	  -p <number>     Partition of image on which / resides
	  -S <number>     Serial port of GCE instance

	  -b <base>       Common path base for files and names
	  -g <path>       Path on image of grub.cfg
	  -i <path>       Local path of source image file
	  -r <path>       Local path of raw image file
	  -a <path>       Local path of tar file

	  -h              Show this help message
	USAGE
}

fail() {
	msg=${1}; errno=${2-1}

	>&2 printf 'Error: %s\n' "${msg}"
	>&2 usage
	exit "${errno}"
}

show_vars() {
	printf 'gce_image_family=%s\n' "${gce_image_family}"
	printf 'gce_image_version=%s\n' "${gce_image_version}"
	printf 'gce_image_name=%s\n' "${gce_image_name}"
	printf '\n'

	printf 'gce_vm_name=%s\n' "${gce_vm_name}"
	printf 'gce_vm_subnet=%s\n' "${gce_vm_subnet}"
	printf 'gce_vm_type=%s\n' "${gce_vm_type}"
	printf '\n'

	printf 'ord_partition=%s\n' "${ord_partition}"
	printf 'ord_serial_port=%s\n' "${ord_serial_port}"
	printf '\n'

	printf 'path_base=%s\n' "${path_base}"
	printf 'path_grub_cfg=%s\n' "${path_grub_cfg}"
	printf 'path_qcow2=%s\n' "${path_qcow2}"
	printf 'path_raw=%s\n' "${path_raw}"
	printf 'path_tar_gz=%s\n' "${path_tar_gz}"
	printf '\n'

	printf 'gs_uri_bucket=%s\n' "${gs_uri_bucket}"
	printf 'gs_uri_tar=%s\n' "${gs_uri_tar}"
}

size_mod_gb() {
	img=${1}
	size=$(stat -c '%s' "${img}")
	mod_gb=$((size / (1024*1024*1024)))
	rem_gb=$((size % (1024*1024*1024)))

	printf '%s %s\n' "${mod_gb}" "${rem_gb}"
}

## GCE requires image sizes to be an integer number of GBs
grow_to_ciel_gbs() {
	img=${1}

	size_mod_gb "${img}" \
	| if read -r sz_mod_gb sz_rem_gb _; then
		[ "${sz_rem_gb}" -eq 0 ] && return
		dd if=/dev/zero of="${img}" \
		   bs=1G count=0 seek=$((sz_mod_gb + 1))
	fi
}

## GCE puts a few contraints on kernel cmdline arguments
#
# Remove unsupported:    `splashimage`, `rhgb`, and `quiet`
# Enable serial console: `console=ttyS0,38400n8d`
prepare_grub_cfg() {
	grub_cfg=${1}

	sudo sed -i -f - "${grub_cfg}" <<-'SED'
		/^\s\+linux/ { s/\s\+\(rhgb\|quiet\|splashimage\)//g }
		/^\s\+linux/ {
			s/\s\+console=ttyS0,38400n8d//g
			s/$/ console=ttyS0,38400n8d/
		}
	SED
}

check_grub_cfg() {
	grub_cfg=${1}

	sudo sed -n -f - "${grub_cfg}" <<-'SED'
		/^\s\+linux.*\(rhgb\|quiet\|splashimage\)/ { p }
		/^\s\+linux/ {
			s/\s\+console=ttyS0,38400n8d/&/g
			t; F; p
		}
	SED
}

free_nbd_dev() {
	sudo modprobe -v nbd
	for nbd in /sys/class/block/nbd*; do
		[ "$(cat "${nbd}"/size)" -ne 0 ] && continue
		dev_block=$(cat "${nbd}/dev")
		readlink -vf "/dev/block/${dev_block}"
		break
	done
}

mount_image_part() {
	img=${1}; part=${2}; mnt=${3}
	dev=$(free_nbd_dev)

	sudo qemu-nbd --format=qcow2 --connect="${dev}" "${img}"
	mkdir -vp "${mnt}"
	sudo mount -v "${dev}p${part}" "${mnt}"
}

umount_image() {
	mnt=${1}
	dev=$(mount | awk "\$3 ~ \"${mnt}\" {print \$1}")

	sudo umount -v "${mnt}"
	rmdir -v "${mnt}"

	sudo qemu-nbd --disconnect "${dev}"
}

configure_files() {
	img=${1}; part=${2}; grub_cfg=${3}
	mnt=$(mktemp -d)

	mount_image_part "${img}" "${part}" "${mnt}"
	prepare_grub_cfg "${mnt}/${grub_cfg}"
	umount_image "${mnt}"
}

check_files() {
	img=${1}; part=${2}; grub_cfg=${3}
	mnt=$(mktemp -d)

	mount_image_part "${img}" "${part}" "${mnt}"
	check_grub_cfg "${mnt}/${grub_cfg}"
	umount_image "${mnt}"
}

## GCE image source expects oldgnu gzip tar with image named `disk.raw`
archive_to_tar_gz() {
	raw=${1}; tar_gz=${2}
	dir=$(mktemp -d)

	ln -vs "${raw}" "${dir}/disk.raw"
	tar --verbose --sparse --dereference --format=oldgnu --create --gzip \
		--directory "${dir}" --file "${tar_gz}" disk.raw

	rm -v "${dir}/disk.raw"
	rmdir -v "${dir}"
}

## To debug boot, we must explicitly enable serial I/O
gce_create_vm() {
	name=${1}; image=${2}; subnet=${3}; type=${4}; port=${5}

	gcloud compute instances create "${name}" \
		--image "${image}" \
		--subnet "${subnet}" \
		--machine-type "${type}"
	gcloud compute instances add-metadata "${name}" \
		--metadata "serial-port-enable=${port}"
}

while getopts ':a:A:b:B:f:g:i:m:n:N:p:r:s:S:v:h' opt "${@}"; do
	case "${opt}" in
		f) opt_gce_image_family=${OPTARG};;
		v) opt_gce_image_version=${OPTARG};;
		n) opt_gce_image_name=${OPTARG};;

		N) opt_gce_vm_name=${OPTARG};;
		s) opt_gce_vm_subnet=${OPTARG};;
		m) opt_gce_vm_type=${OPTARG};;

		B) opt_gs_uri_bucket=${OPTARG};;
		A) opt_gs_uri_tar=${OPTARG};;

		p) opt_ord_partition=${OPTARG};;
		S) opt_ord_serial_port=${OPTARG};;

		b) opt_path_base=${OPTARG};;
		g) opt_path_grub_cfg=${OPTARG};;
		i) opt_path_qcow2=${OPTARG};;
		r) opt_path_raw=${OPTARG};;
		a) opt_path_tar_gz=${OPTARG};;

		h) usage; exit;;
		:) fail "Expected argument to option -${OPTARG}";;
		*) fail "Unrecognized option: -${OPTARG}";;
	esac
done
shift $((OPTIND - 1))


gce_image_family=${opt_gce_image_family:-gce-vm}
gce_image_version=${opt_gce_image_version:-}
gce_image_name=${opt_gce_image_name:-${gce_image_family}${gce_image_version:+-}${gce_image_version}}

gce_vm_name=${opt_gce_vm_name:-${gce_image_name}}
gce_vm_subnet=${opt_gce_vm_subnet:-default}
gce_vm_type=${opt_gce_vm_type:-f1-micro}

ord_partition=${opt_ord_partition:-1}
ord_serial_port=${opt_ord_serial_port:-1}

path_grub_cfg=${opt_path_grub_cfg:-/boot/grub/grub.cfg}
path_base=${opt_path_base:-${PWD}/imgs/${gce_image_name}}
path_qcow2=${opt_path_qcow2:-${path_base}.qcow2}
path_raw=${opt_path_raw:-${path_base}.raw}
path_tar_gz=${opt_path_tar_gz:-${path_base}.tar.gz}

gs_uri_bucket=${opt_gs_uri_bucket:-gs://$(date +%s.%N)}
gs_uri_tar=${opt_gs_uri_tar:-${gs_uri_bucket}/$(basename "${path_tar_gz}")}


[ $# -gt 0 ] || fail "Expected command"

for command in "${@}"; do
	case "${command}" in
		# Setup
		mkconf) configure_files "${path_qcow2}" \
		                        "${ord_partition}" \
		                        "${path_grub_cfg}";;
		mkraw)  qemu-img convert -O raw "${path_qcow2}" \
		                                "${path_raw}";;
		mksize) grow_to_ciel_gbs "${path_raw}";;
		mktar)  archive_to_tar_gz "${path_raw}" \
		                          "${path_tar_gz}";;
		mkgsb)  gsutil mb "${gs_uri_bucket}";;
		togsa)  gsutil cp "${path_tar_gz}" "${gs_uri_bucket}";;
		togci)  gcloud compute images create "${gce_image_name}" \
		               --source-uri "${gs_uri_tar}" \
		               --family "${gce_image_family}";;
		togvm)  gce_create_vm "${gce_vm_name}" \
		                      "${gce_image_name}" \
		                      "${gce_vm_subnet}" \
		                      "${gce_vm_type}" \
		                      "${ord_serial_port}";;
		ttysro) gcloud compute instances get-serial-port-output \
		               "${gce_vm_name}" \
		               --port "${ord_serial_port}";;
		ttysrw) gcloud compute connect-to-serial-port \
		               "${gce_vm_name}" \
		               --port "${ord_serial_port}";;

		# Status
		ckconf) check_files "${path_qcow2}" \
		                    "${ord_partition}" \
		                    "${path_grub_cfg}";;
		ckraw)  qemu-img compare "${path_qcow2}" "${path_raw}";;
		cksize) size_mod_gb "${path_raw}" | awk '$2 != 0 {exit 1}';;
		cktar)  tar -tf "${path_tar_gz}" | sed '/^disk.raw$/!q1';;
		ckgsb)  gsutil du "${gs_uri_bucket}";;
		ckgsa)  gsutil stat "${gs_uri_tar}";;
		ckgci)  gcloud compute images describe "${gce_image_name}";;
		ckgvm)  gcloud compute instances describe "${gce_vm_name}";;

		# Removal
		rmgvm)  gcloud compute instances delete "${gce_vm_name}";;
		rmgci)  gcloud compute images delete "${gce_image_name}";;
		rmgsa)  gsutil rm "${gs_uri_tar}";;
		rmgsb)  gsutil rb "${gs_uri_tar}";;

		# Metainfo
		help)   usage;;
		vars)   show_vars;;

		*) fail "Unrecognized command: ${command}";;
	esac
done

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Guix System on a GCE VM -- Success

[Jelle Licht]

Hey there,

elaexuotee@wilsonb.com writes:

> To simplify the process I created a script capable of setting up a GCE instance
> from a local qcow2 image, which I am also attaching. The script is intentially
> not a simple turnkey solution, since you should be somewhat familiar with the
> GCE infrastructure before using it. However, feel free to ask questions if
> anything is unclear.

This all looks pretty nifty! Just to be clear, under which license did
you make this script available?

Thanks!
- Jelle

Re: Guix System on a GCE VM -- Success

[Jonathan Brielmaier]

On 08.04.20 05:16, elaexuotee@wilsonb.com wrote:
> The boot problem fix boils down to this:
>
>     (initrd-modules (cons "virtio_scsi" %base-initrd-modules))
>
> Referencing Google's documentation on operating system requirements [2], we see
> that their VMs run with Virtio-SCSI controllers; however, it just so happens
> that the corresponding kernel module, virtio_scsi, is not included in the
> initrd by default. Would it make sense to change this default?

This had to been done on Hetzner Cloud as well. I wonder if its not
fixed already on master. Need to check that...

Re: Guix System on a GCE VM -- Success

[elaexuotee--- via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1.1: Type: text/plain, Size: 299 bytes --]

> This all looks pretty nifty! Just to be clear, under which license did
> you make this script available?

Ouch. That was a major oversight. Thanks for bringing this up!
I went ahead and put it under the BSD-3-CLAUSE. Attached is the same source
with a copyright notice and license header.


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Guix System on a GCE VM -- Success

[elaexuotee]

[-- Attachment #1.1: Type: text/plain, Size: 478 bytes --]

elaexuotee--- via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> wrote:
> > This all looks pretty nifty! Just to be clear, under which license did
> > you make this script available?
> 
> Ouch. That was a major oversight. Thanks for bringing this up!
> I went ahead and put it under the BSD-3-CLAUSE. Attached is the same source
> with a copyright notice and license header.


It looks like I messed up the attachment. Trying again.


[-- Attachment #1.2: gce.sh --]
[-- Type: text/plain, Size: 11735 bytes --]

#!/usr/bin/env sh
set -o errexit -o noclobber -o nounset

# Copyright (c) 2020 elaexuotee@wilsonb.com. All rights reserved.
#
# Licensed under <https://spdx.org/licenses/BSD-3-Clause.html>
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
#    this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors
#    may be used to endorse or promote products derived from this software
#    without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.


## GCE Custom Image
#
# This script intends to be "executable documention" for the process of setting
# up a Google Compute Engine (GCE) virtual machine from a custom image.


usage() {
	cat <<-USAGE
	Usage: $(basename "${0}") [options] <command> [<command> [..]]

	This script aids in setting up Google Compute Engine with an instance
	running a custom image.

	Execute commands in order provided at the command line. The procedure
	for a fresh setup follows the order listed below in Status Commands.

	Setup Commands:
	  mkconf          Configure files on raw image partition
	  mkraw           Convert image to raw format
	  mksize          Grow image to integer multiple of GBs
	  mktar           Archive raw image
	  mkgsb           Create GS bucket

	  togsa           Upload archive to GS bucket
	  togci           Create GCE image from GS archive
	  togvm           Create GCE instance from GCE image

	Status Commands:
	  ckconf          Check contents of qcow2 image
	  ckraw           Check integrity of raw image
	  cksize          Check size of raw image
	  cktar           Check contents of tar archive
	  ckgsb           Check existence of GS bucket
	  ckgsa           Check existence of tar archive in GS bucket
	  ckgci           Check existence of GCE image
	  ckgvm           Check status of GCE instance

	  ttysro          View GCE instance's serial output
	  ttysrw          Connect to instance serial port

	Removal Commands:
	  rmgvm           Remove GCE instance
	  rmgci           Remove GCE image
	  rmgsa           Remove archive from GS bucket
	  rmgsb           Remove GS bucket

	Metainfo Commands:
	  help            Show this help message
	  vars            Print value of configuration variables

	Options:
	  -f <family>     Family name for GCE image
	  -v <version>    Version string of GCE image
	  -n <name>       Name of GCE image

	  -N <name>       Name of GCE instance
	  -s <subnet>     Subnet name of GCE instance
	  -m <type>       Machine type of GCE instance

	  -B <uri>        URI of GS bucket
	  -A <uri>        URI of GS tar file

	  -p <number>     Partition of image on which / resides
	  -S <number>     Serial port of GCE instance

	  -b <base>       Common path base for files and names
	  -g <path>       Path on image of grub.cfg
	  -i <path>       Local path of source image file
	  -r <path>       Local path of raw image file
	  -a <path>       Local path of tar file

	  -h              Show this help message
	USAGE
}

fail() {
	msg=${1}; errno=${2-1}

	>&2 printf 'Error: %s\n' "${msg}"
	>&2 usage
	exit "${errno}"
}

show_vars() {
	printf 'gce_image_family=%s\n' "${gce_image_family}"
	printf 'gce_image_version=%s\n' "${gce_image_version}"
	printf 'gce_image_name=%s\n' "${gce_image_name}"
	printf '\n'

	printf 'gce_vm_name=%s\n' "${gce_vm_name}"
	printf 'gce_vm_subnet=%s\n' "${gce_vm_subnet}"
	printf 'gce_vm_type=%s\n' "${gce_vm_type}"
	printf '\n'

	printf 'ord_partition=%s\n' "${ord_partition}"
	printf 'ord_serial_port=%s\n' "${ord_serial_port}"
	printf '\n'

	printf 'path_base=%s\n' "${path_base}"
	printf 'path_grub_cfg=%s\n' "${path_grub_cfg}"
	printf 'path_qcow2=%s\n' "${path_qcow2}"
	printf 'path_raw=%s\n' "${path_raw}"
	printf 'path_tar_gz=%s\n' "${path_tar_gz}"
	printf '\n'

	printf 'gs_uri_bucket=%s\n' "${gs_uri_bucket}"
	printf 'gs_uri_tar=%s\n' "${gs_uri_tar}"
}

size_mod_gb() {
	img=${1}
	size=$(stat -c '%s' "${img}")
	mod_gb=$((size / (1024*1024*1024)))
	rem_gb=$((size % (1024*1024*1024)))

	printf '%s %s\n' "${mod_gb}" "${rem_gb}"
}

## GCE requires image sizes to be an integer number of GBs
grow_to_ciel_gbs() {
	img=${1}

	size_mod_gb "${img}" \
	| if read -r sz_mod_gb sz_rem_gb _; then
		[ "${sz_rem_gb}" -eq 0 ] && return
		dd if=/dev/zero of="${img}" \
		   bs=1G count=0 seek=$((sz_mod_gb + 1))
	fi
}

## GCE puts a few contraints on kernel cmdline arguments
#
# Remove unsupported:    `splashimage`, `rhgb`, and `quiet`
# Enable serial console: `console=ttyS0,38400n8d`
prepare_grub_cfg() {
	grub_cfg=${1}

	sudo sed -i -f - "${grub_cfg}" <<-'SED'
		/^\s\+linux/ { s/\s\+\(rhgb\|quiet\|splashimage\)//g }
		/^\s\+linux/ {
			s/\s\+console=ttyS0,38400n8d//g
			s/$/ console=ttyS0,38400n8d/
		}
	SED
}

check_grub_cfg() {
	grub_cfg=${1}

	sudo sed -n -f - "${grub_cfg}" <<-'SED'
		/^\s\+linux.*\(rhgb\|quiet\|splashimage\)/ { p }
		/^\s\+linux/ {
			s/\s\+console=ttyS0,38400n8d/&/g
			t; F; p
		}
	SED
}

free_nbd_dev() {
	sudo modprobe -v nbd
	for nbd in /sys/class/block/nbd*; do
		[ "$(cat "${nbd}"/size)" -ne 0 ] && continue
		dev_block=$(cat "${nbd}/dev")
		readlink -vf "/dev/block/${dev_block}"
		break
	done
}

mount_image_part() {
	img=${1}; part=${2}; mnt=${3}
	dev=$(free_nbd_dev)

	sudo qemu-nbd --format=qcow2 --connect="${dev}" "${img}"
	mkdir -vp "${mnt}"
	sudo mount -v "${dev}p${part}" "${mnt}"
}

umount_image() {
	mnt=${1}
	dev=$(mount | awk "\$3 ~ \"${mnt}\" {print \$1}")

	sudo umount -v "${mnt}"
	rmdir -v "${mnt}"

	sudo qemu-nbd --disconnect "${dev}"
}

configure_files() {
	img=${1}; part=${2}; grub_cfg=${3}
	mnt=$(mktemp -d)

	mount_image_part "${img}" "${part}" "${mnt}"
	prepare_grub_cfg "${mnt}/${grub_cfg}"
	umount_image "${mnt}"
}

check_files() {
	img=${1}; part=${2}; grub_cfg=${3}
	mnt=$(mktemp -d)

	mount_image_part "${img}" "${part}" "${mnt}"
	check_grub_cfg "${mnt}/${grub_cfg}"
	umount_image "${mnt}"
}

## GCE image source expects oldgnu gzip tar with image named `disk.raw`
archive_to_tar_gz() {
	raw=${1}; tar_gz=${2}
	dir=$(mktemp -d)

	ln -vs "${raw}" "${dir}/disk.raw"
	tar --verbose --sparse --dereference --format=oldgnu --create --gzip \
		--directory "${dir}" --file "${tar_gz}" disk.raw

	rm -v "${dir}/disk.raw"
	rmdir -v "${dir}"
}

## To debug boot, we must explicitly enable serial I/O
gce_create_vm() {
	name=${1}; image=${2}; subnet=${3}; type=${4}; port=${5}

	gcloud compute instances create "${name}" \
		--image "${image}" \
		--subnet "${subnet}" \
		--machine-type "${type}"
	gcloud compute instances add-metadata "${name}" \
		--metadata "serial-port-enable=${port}"
}

while getopts ':a:A:b:B:f:g:i:m:n:N:p:r:s:S:v:h' opt "${@}"; do
	case "${opt}" in
		f) opt_gce_image_family=${OPTARG};;
		v) opt_gce_image_version=${OPTARG};;
		n) opt_gce_image_name=${OPTARG};;

		N) opt_gce_vm_name=${OPTARG};;
		s) opt_gce_vm_subnet=${OPTARG};;
		m) opt_gce_vm_type=${OPTARG};;

		B) opt_gs_uri_bucket=${OPTARG};;
		A) opt_gs_uri_tar=${OPTARG};;

		p) opt_ord_partition=${OPTARG};;
		S) opt_ord_serial_port=${OPTARG};;

		b) opt_path_base=${OPTARG};;
		g) opt_path_grub_cfg=${OPTARG};;
		i) opt_path_qcow2=${OPTARG};;
		r) opt_path_raw=${OPTARG};;
		a) opt_path_tar_gz=${OPTARG};;

		h) usage; exit;;
		:) fail "Expected argument to option -${OPTARG}";;
		*) fail "Unrecognized option: -${OPTARG}";;
	esac
done
shift $((OPTIND - 1))


gce_image_family=${opt_gce_image_family:-gce-vm}
gce_image_version=${opt_gce_image_version:-}
gce_image_name=${opt_gce_image_name:-${gce_image_family}${gce_image_version:+-}${gce_image_version}}

gce_vm_name=${opt_gce_vm_name:-${gce_image_name}}
gce_vm_subnet=${opt_gce_vm_subnet:-default}
gce_vm_type=${opt_gce_vm_type:-f1-micro}

ord_partition=${opt_ord_partition:-1}
ord_serial_port=${opt_ord_serial_port:-1}

path_grub_cfg=${opt_path_grub_cfg:-/boot/grub/grub.cfg}
path_base=${opt_path_base:-${PWD}/imgs/${gce_image_name}}
path_qcow2=${opt_path_qcow2:-${path_base}.qcow2}
path_raw=${opt_path_raw:-${path_base}.raw}
path_tar_gz=${opt_path_tar_gz:-${path_base}.tar.gz}

gs_uri_bucket=${opt_gs_uri_bucket:-gs://$(date +%s.%N)}
gs_uri_tar=${opt_gs_uri_tar:-${gs_uri_bucket}/$(basename "${path_tar_gz}")}


[ $# -gt 0 ] || fail "Expected command"

for command in "${@}"; do
	case "${command}" in
		# Setup
		mkconf) configure_files "${path_qcow2}" \
		                        "${ord_partition}" \
		                        "${path_grub_cfg}";;
		mkraw)  qemu-img convert -O raw "${path_qcow2}" \
		                                "${path_raw}";;
		mksize) grow_to_ciel_gbs "${path_raw}";;
		mktar)  archive_to_tar_gz "${path_raw}" \
		                          "${path_tar_gz}";;
		mkgsb)  gsutil mb "${gs_uri_bucket}";;
		togsa)  gsutil cp "${path_tar_gz}" "${gs_uri_bucket}";;
		togci)  gcloud compute images create "${gce_image_name}" \
		               --source-uri "${gs_uri_tar}" \
		               --family "${gce_image_family}";;
		togvm)  gce_create_vm "${gce_vm_name}" \
		                      "${gce_image_name}" \
		                      "${gce_vm_subnet}" \
		                      "${gce_vm_type}" \
		                      "${ord_serial_port}";;
		ttysro) gcloud compute instances get-serial-port-output \
		               "${gce_vm_name}" \
		               --port "${ord_serial_port}";;
		ttysrw) gcloud compute connect-to-serial-port \
		               "${gce_vm_name}" \
		               --port "${ord_serial_port}";;

		# Status
		ckconf) check_files "${path_qcow2}" \
		                    "${ord_partition}" \
		                    "${path_grub_cfg}";;
		ckraw)  qemu-img compare "${path_qcow2}" "${path_raw}";;
		cksize) size_mod_gb "${path_raw}" | awk '$2 != 0 {exit 1}';;
		cktar)  tar -tf "${path_tar_gz}" | sed '/^disk.raw$/!q1';;
		ckgsb)  gsutil du "${gs_uri_bucket}";;
		ckgsa)  gsutil stat "${gs_uri_tar}";;
		ckgci)  gcloud compute images describe "${gce_image_name}";;
		ckgvm)  gcloud compute instances describe "${gce_vm_name}";;

		# Removal
		rmgvm)  gcloud compute instances delete "${gce_vm_name}";;
		rmgci)  gcloud compute images delete "${gce_image_name}";;
		rmgsa)  gsutil rm "${gs_uri_tar}";;
		rmgsb)  gsutil rb "${gs_uri_tar}";;

		# Metainfo
		help)   usage;;
		vars)   show_vars;;

		*) fail "Unrecognized command: ${command}";;
	esac
done

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 260 bytes --]