Using the Hetzner Cloud

Using the Hetzner Cloud

[Jonathan Brielmaier]

Hi folks,

as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they
could provide us some free VMs in their cloud[1].

A few days ago they came back go to me. Sadly they can't provide us free
VMs, but they will give us 30 EUR credit for the first month.

As they bill hour-wise and offer a serial console over HTTPS, you can do
tests for stuff like the installer for 2 or 3 cents :)

What do you think about it?

~Jonathan

P.S: I encourage you to ask your local/well-known hosting provider for
some support in any way for Guix. It's not so much effort.

[0] https://www.hetzner.de/ big German hosting company
[1] https://www.hetzner.de/cloud

Re: Using the Hetzner Cloud

[Ellen Papsch]

Hi,

Am Montag, den 17.02.2020, 14:47 +0100 schrieb Jonathan Brielmaier:
> Hi folks,
> 
> as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they
> could provide us some free VMs in their cloud[1].
> 
> A few days ago they came back go to me. Sadly they can't provide us
> free
> VMs, but they will give us 30 EUR credit for the first month.
> 
> As they bill hour-wise and offer a serial console over HTTPS, you can
> do
> tests for stuff like the installer for 2 or 3 cents :)
> 
> What do you think about it?
> 
> ~Jonathan
> 

a few days ago I requested the Guix installer from Hetzner as well. I
gave them the ISO URL from Guix website. They mounted it without
modification, which prevented boot due to the image being compressed. I
had to supply the uncompressed image, which booted fine.

The installer was not able to install right away. I could complete all
the GUI steps. When it came to guix system init, it bailed with the
message that an initrd-module may be missing. System init (and the
subsequent boot) worked fine with:

(initrd-modules (append (list "virtio_pci"
                              "virtio_scsi")
                        %base-initrd-modules))

BTW, the VGA console is very limited, you can't select text or send
special key strokes, e.g. control keys. It suffices for activating SSH
though :-)

Hetzner added the image only for me, which is a hindrance for Guix
adoption. Another limitation is that I can't create new servers right
away without going through installation. I can dance around it by
creating a snapshot of a template server and creating the new server
from that snapshot, then reconfiguring the system (or doing a deploy, I
want to try that yet).

So, doing Guix is a little more inconvenient than a simple, plain
Debian. I can still recommend it, though you may find Hetzner not
meeting your demands in general: The cloud offerings have a rather low
cpu/memory ratio (0.25 cpu/1G RAM) and it's not configurable.

Best regards
Ellen

Re: Using the Hetzner Cloud

[Alex Sassmannshausen]

Hello,

Ellen Papsch <ellen.papsch@wine-logistix.de> writes:

> Hi,
>
> Am Montag, den 17.02.2020, 14:47 +0100 schrieb Jonathan Brielmaier:
>> Hi folks,
>> 
>> as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they
>> could provide us some free VMs in their cloud[1].
>>
>> […]
>> 
>> What do you think about it?
>> 
>> ~Jonathan
>> 
>
> a few days ago I requested the Guix installer from Hetzner as well. I
> […]
> cpu/memory ratio (0.25 cpu/1G RAM) and it's not configurable.
>
> Best regards
> Ellen

Figured I might as well add my experiences.

I originally have done the same thing as you did Ellen.  This was a bit
of a slow process for deploying Guix to Hetzner servers though.

Now I use a different approach: deploy a debian server then use a
guix-infect style script (gleaned from the guix deploy code for digital
ocean).

So I deploy debian, then copy across a script and run that.  This takes
care of turning the debian machine into a guix machine and deploys my
sys config immediately.

This is by far the fastest way of deploying to Hetzner VMs I've found
yet.

It also means you can use Hetzner's CLI for creating and managing your
VMs.

It's on my to do list to create a solid guix deploy server creation
module for guix.

Hth,

Alex

Re: Using the Hetzner Cloud

[Ellen Papsch]

Am Montag, den 17.02.2020, 18:15 +0100 schrieb Alex Sassmannshausen:
> 
> 
> So I deploy debian, then copy across a script and run that.  This
> takes
> care of turning the debian machine into a guix machine and deploys my
> sys config immediately.
> 
> This is by far the fastest way of deploying to Hetzner VMs I've found
> yet.
> 
> It also means you can use Hetzner's CLI for creating and managing
> your
> VMs.

That sounds nice. I had wondered if there's some sort of FrankenDebian
way but did not pursue it. Given this line in the script:

mv var/guix /var/ && mv gnu /

there seems to be a complete takeover, even better than a FrankenDebian
:-)


> It's on my to do list to create a solid guix deploy server creation
> module for guix.

That would certainly be appreciated.

I'm currently in the process of cementing Guix usage here (for
databases at least). If I succeed and get some time, I might be able to
do a first stab. Though it would be more useful for applications, which
come and go more often than databases. We currently manage apps with
ansible, I already have started lobbying at low rate. But first things
first -- databases!

Best regards

Re: Using the Hetzner Cloud

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

Hello Alex,

Alex Sassmannshausen <alex.sassmannshausen@gmail.com> writes:

[...]

> Now I use a different approach: deploy a debian server then use a
> guix-infect style script (gleaned from the guix deploy code for digital
> ocean).

could you please share that "guix-infect" script?

I think this could/should become an entry in our cookbook, in a similar
way NixOS does here: https://nixos.wiki/wiki/NixOS_friendly_hosters#Hoster-agnostic_means_of_installation

[...]

> It's on my to do list to create a solid guix deploy server creation
> module for guix.

that's the long term solution and it'll be fantastic, IMHO it's still
interesting to show a "hoster agnostic means of installation" for Guix

Thanks! Gio'

[...]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Using the Hetzner Cloud

[Alex Sassmannshausen]

[-- Attachment #1: Type: text/plain, Size: 1429 bytes --]

Heya,

Giovanni Biscuolo <g@xelera.eu> writes:

> Hello Alex,
>
> Alex Sassmannshausen <alex.sassmannshausen@gmail.com> writes:
>
> [...]
>
>> Now I use a different approach: deploy a debian server then use a
>> guix-infect style script (gleaned from the guix deploy code for digital
>> ocean).
>
> could you please share that "guix-infect" script?

Sure, please see attached.  This one here is a bash script that works
with a system config that is specified as part of a here-doc in the bash
script.

It's all a bit gaffer tape & macgyver — primarily to encourage myself to
keep automating further by integrating this in Guix deploy. Where does
all the time go!!!

Happy to help if you run into problems or have questions.

> I think this could/should become an entry in our cookbook, in a similar
> way NixOS does here:
> https://nixos.wiki/wiki/NixOS_friendly_hosters#Hoster-agnostic_means_of_installation

Agreed, a cookbook recipe that summarises the existing ways of deploying
would be cool.  I'd be happy to read over and give feedback on any
proposed articles.

Ellen Papsch <ellen.papsch@wine-logistix.de> writes:

> […]
> mv var/guix /var/ && mv gnu /
>
> there seems to be a complete takeover, even better than a FrankenDebian
> :-)

Agreed — it's testament to the versatility of Guix that it can literally
do this hostile take-over.  Very cool :-)

Cheers,

Alex

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: boostrap.sh --]
[-- Type: text/x-sh, Size: 3936 bytes --]

#!/bin/bash

e2label /dev/sda1 root
apt-get update
apt-get install xz-utils -y
wget https://ftp.gnu.org/gnu/guix/guix-binary-1.0.1.x86_64-linux.tar.xz
cd /tmp
tar --warning=no-timestamp -xf ~/guix-binary-1.0.1.x86_64-linux.tar.xz
mv var/guix /var/ && mv gnu /
mkdir -p ~root/.config/guix
ln -sf /var/guix/profiles/per-user/root/current-guix ~root/.config/guix/current
export GUIX_PROFILE="`echo ~root`/.config/guix/current" ;
source $GUIX_PROFILE/etc/profile
groupadd --system guixbuild
for i in `seq -w 1 10`; do
   useradd -g guixbuild -G guixbuild                    -d /var/empty -s `which nologin`             -c "Guix build user $i" --system             guixbuilder$i;
done;
cp ~root/.config/guix/current/lib/systemd/system/guix-daemon.service /etc/systemd/system/
systemctl start guix-daemon && systemctl enable guix-daemon
mkdir -p /usr/local/bin
cd /usr/local/bin
ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix
mkdir -p /usr/local/share/info
cd /usr/local/share/info
for i in /var/guix/profiles/per-user/root/current-guix/share/info/*; do
    ln -s $i;
done
guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub
# FIXME: I'm pulling a commit that fixes some issues.  When there is a new
# guix release this can be removed.
guix pull --commit=3a695c01d7ee18f30f22df53f3c44dfac04017f1
guix package -i openssl
# FIXME: Just loading the default example from the guix manual here.  This can
# be adapted to whatever base guix deployment you want.
cat > /etc/bootstrap-config.scm << EOF
     ;; This is an operating system configuration template
     ;; for a "bare bones" setup, with no X11 display server.

     (use-modules (gnu))
     (use-service-modules networking ssh)
     (use-package-modules screen)

     (operating-system
       (host-name "komputilo")
       (timezone "Europe/Berlin")
       (locale "en_US.utf8")

       ;; Boot in "legacy" BIOS mode, assuming /dev/sdX is the
       ;; target hard disk, and "my-root" is the label of the target
       ;; root file system.
       (bootloader (bootloader-configuration
                     (bootloader grub-bootloader)
                     (target "/dev/sdX")))
       (file-systems (cons (file-system
                             (device (file-system-label "my-root"))
                             (mount-point "/")
                             (type "ext4"))
                           %base-file-systems))

       ;; This is where user accounts are specified.  The "root"
       ;; account is implicit, and is initially created with the
       ;; empty password.
       (users (cons (user-account
                     (name "alice")
                     (comment "Bob's sister")
                     (group "users")

                     ;; Adding the account to the "wheel" group
                     ;; makes it a sudoer.  Adding it to "audio"
                     ;; and "video" allows the user to play sound
                     ;; and access the webcam.
                     (supplementary-groups '("wheel"
                                             "audio" "video")))
                    %base-user-accounts))

       ;; Globally-installed packages.
       (packages (cons screen %base-packages))

       ;; Add services to the baseline: a DHCP client and
       ;; an SSH server.
       (services (append (list (service dhcp-client-service-type)
                               (service openssh-service-type
                                        (openssh-configuration
                                         (port-number 2222))))
                         %base-services)))
EOF
guix system build /etc/bootstrap-config.scm
guix system reconfigure /etc/bootstrap-config.scm
mv /etc /old-etc
mkdir /etc
cp -r /old-etc/{passwd,group,shadow,gshadow,mtab,guix,bootstrap-config.scm} /etc/
guix system reconfigure /etc/bootstrap-config.scm
echo ". /etc/bashrc" >> /root/.bashrc
echo ". /etc/profile" >> /root/.bashrc

Re: Using the Hetzner Cloud

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1364 bytes --]

Hello Alex,

Alex Sassmannshausen <alex.sassmannshausen@gmail.com> writes:

> Giovanni Biscuolo <g@xelera.eu> writes:

[...]

>> could you please share that "guix-infect" script?
>
> Sure, please see attached.  This one here is a bash script that works
> with a system config that is specified as part of a here-doc in the bash
> script.

Cool, thanks!

[...]

> — primarily to encourage myself to keep automating further by
> integrating this in Guix deploy. Where does all the time go!!!

I'd like to help but still need to enhance my guile-foo

> Happy to help if you run into problems or have questions.

maybe when guix will be in Debian [1] your script could be simpified in
the first steps... but I hope we'll have a proper guix-deploy way for
many many providers before :-)

just one more question:

[...]

> #!/bin/bash
>
> e2label /dev/sda1 root

[...]

> guix system build /etc/bootstrap-config.scm
> guix system reconfigure /etc/bootstrap-config.scm
> mv /etc /old-etc
> mkdir /etc
> cp -r /old-etc/{passwd,group,shadow,gshadow,mtab,guix,bootstrap-config.scm} /etc/
> guix system reconfigure /etc/bootstrap-config.scm

why two "guix system reconfigure"?

[...]

Thanks! Gio'



[1]  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Using the Hetzner Cloud

[Alex Sassmannshausen]

Heya,

Giovanni Biscuolo <g@xelera.eu> writes:

> Hello Alex,
>
> Alex Sassmannshausen <alex.sassmannshausen@gmail.com> writes:
>
>> Giovanni Biscuolo <g@xelera.eu> writes:
>
> [...]
>
>>> could you please share that "guix-infect" script?
>>
>> Sure, please see attached.  This one here is a bash script that works
>> with a system config that is specified as part of a here-doc in the bash
>> script.
>
> Cool, thanks!
>
> [...]
>
>> — primarily to encourage myself to keep automating further by
>> integrating this in Guix deploy. Where does all the time go!!!
>
> I'd like to help but still need to enhance my guile-foo
>
>> Happy to help if you run into problems or have questions.
>
> maybe when guix will be in Debian [1] your script could be simpified in
> the first steps... but I hope we'll have a proper guix-deploy way for
> many many providers before :-)
>
> just one more question:
>
> [...]
>
>> #!/bin/bash
>>
>> e2label /dev/sda1 root
>
> [...]
>
>> guix system build /etc/bootstrap-config.scm
>> guix system reconfigure /etc/bootstrap-config.scm
>> mv /etc /old-etc
>> mkdir /etc
>> cp -r /old-etc/{passwd,group,shadow,gshadow,mtab,guix,bootstrap-config.scm} /etc/
>> guix system reconfigure /etc/bootstrap-config.scm
>
> why two "guix system reconfigure"?

Tbh, not 100% sure.  This script is heavily copypasta.  It looks like
there is an element of transferring debian files away over the course of
the 2, so perhaps it's to fully transition.

But, tbh, not really investigated it, as the second reconfigure is dirt
cheap, and the script just works (tm).

Feel free to experiment!  If you find it's unnecessary, I'd be happy to
hear about it!

Cheers,

Alex

Re: Using the Hetzner Cloud

[Jonathan Brielmaier]

On 17.02.20 18:15, Alex Sassmannshausen wrote:> This is by far the
fastest way of deploying to Hetzner VMs I've found
> yet.
>
> It also means you can use Hetzner's CLI for creating and managing your
> VMs.
>
> It's on my to do list to create a solid guix deploy server creation
> module for guix.

That would be pretty neat!

Re: Using the Hetzner Cloud

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]


Ellen Papsch <ellen.papsch@wine-logistix.de> writes:

> a few days ago I requested the Guix installer from Hetzner as well. I
> gave them the ISO URL from Guix website. They mounted it without
> modification, which prevented boot due to the image being compressed. I
> had to supply the uncompressed image, which booted fine.

I had exactly the same experience, I've had this with other services as
well. For the next release at least, providing there are no big
technical blockers, it would be good to publish an uncompressed ISO
image, as well as a compressed one.

> Hetzner added the image only for me, which is a hindrance for Guix
> adoption. Another limitation is that I can't create new servers right
> away without going through installation. I can dance around it by
> creating a snapshot of a template server and creating the new server
> from that snapshot, then reconfiguring the system (or doing a deploy, I
> want to try that yet).

Hopefully if enough people ask them, they might include it in the ISO
images they provide to everyone.

Thanks for sharing your experiences,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Using the Hetzner Cloud

[Jonathan Brielmaier]

On 17.02.20 17:55, Ellen Papsch wrote:> a few days ago I requested the
Guix installer from Hetzner as well. I
> gave them the ISO URL from Guix website. They mounted it without
> modification, which prevented boot due to the image being compressed. I
> had to supply the uncompressed image, which booted fine.
>
> The installer was not able to install right away. I could complete all
> the GUI steps. When it came to guix system init, it bailed with the
> message that an initrd-module may be missing. System init (and the
> subsequent boot) worked fine with:
>
> (initrd-modules (append (list "virtio_pci"
>                               "virtio_scsi")
>                         %base-initrd-modules))

AFAIK it's fixed in Guix master. But 1.0.1 is missing the patch. It's
time for new release :)

> BTW, the VGA console is very limited, you can't select text or send
> special key strokes, e.g. control keys. It suffices for activating SSH
> though :-)

The console is limited but was always good enough for my use cases :)

> Hetzner added the image only for me, which is a hindrance for Guix
> adoption. Another limitation is that I can't create new servers right
> away without going through installation. I can dance around it by
> creating a snapshot of a template server and creating the new server
> from that snapshot, then reconfiguring the system (or doing a deploy, I
> want to try that yet).
>
> So, doing Guix is a little more inconvenient than a simple, plain
> Debian. I can still recommend it, though you may find Hetzner not
> meeting your demands in general: The cloud offerings have a rather low
> cpu/memory ratio (0.25 cpu/1G RAM) and it's not configurable.

Going through the installer all the time if you have more servers is
annoying. But for smaller setups with one or two servers pretty okay.

Re: Using the Hetzner Cloud

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1385 bytes --]


Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:

> as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they
> could provide us some free VMs in their cloud[1].
>
> A few days ago they came back go to me. Sadly they can't provide us free
> VMs, but they will give us 30 EUR credit for the first month.
>
> As they bill hour-wise and offer a serial console over HTTPS, you can do
> tests for stuff like the installer for 2 or 3 cents :)
>
> What do you think about it?

Thanks for looking in to the Jonathan, that's really useful.

I've been using Hetzner for hosting data.guix.gnu.org for at least the
last few months now. I've been paying for it so far, but ideally it
wouldn't depend on me personally paying for it in the long term.

Coupled with this, I think there are some services (hpc.guix.info, ...)
it would be nice to move off bayfront (a machine in Bordeaux) to
somewhere that's more reliable/remotely accessible, mostly to avoid the
risk of me breaking them again when attempting to reconfigure bayfront.

Maybe Guix Europe (a legal entity in Europe) could have a Hetzner
account that could be used at the discretion of those involved in Guix
Europe? I think Hetzner support billing by SEPA direct debit, which
might work, otherwise I can probably pay the bill and then be
reimbursed.

Thanks again for following up talking to Hetzner Jonathan!

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Using the Hetzner Cloud

[Ricardo Wurmus]

Christopher Baines <mail@cbaines.net> writes:

> Coupled with this, I think there are some services (hpc.guix.info, ...)
> it would be nice to move off bayfront (a machine in Bordeaux) to
> somewhere that's more reliable/remotely accessible, mostly to avoid the
> risk of me breaking them again when attempting to reconfigure bayfront.

That’s unfortunate.  Why is bayfront less reliable than we want it to
be?  Surely that’s not because it’s using Guix System, because
{berlin,ci}.guix.gnu.org also runs Guix System and has been reliably
hosting services for us.

I would not mind hosting services on one of the 30 new servers the MDC
bought for ci.guix.gnu.org, but I’d be happier if we figured out what’s
wrong with bayfront instead of centralizing all services in Berlin…

-- 
Ricardo

Re: Using the Hetzner Cloud

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]


Ricardo Wurmus <rekado@elephly.net> writes:

> Christopher Baines <mail@cbaines.net> writes:
>
>> Coupled with this, I think there are some services (hpc.guix.info, ...)
>> it would be nice to move off bayfront (a machine in Bordeaux) to
>> somewhere that's more reliable/remotely accessible, mostly to avoid the
>> risk of me breaking them again when attempting to reconfigure bayfront.
>
> That’s unfortunate.  Why is bayfront less reliable than we want it to
> be?  Surely that’s not because it’s using Guix System, because
> {berlin,ci}.guix.gnu.org also runs Guix System and has been reliably
> hosting services for us.

I guess having access to recover from issues is the primary concern, and
reliability relates to that.

Around FOSDEM, I think I caused Bayfront to restart while attempting to
restart cuirass. I'm not sure exactly what happened, as my SSH
connection just dropped, and the machine became inaccessible (including
Cuirass, hpc.guix.info, ...).

Luckily, some time later Ludovic and Andreas were available to remotely
access the power supply, and turn the machine off and on, this did seem
to bring the machine back.

> I would not mind hosting services on one of the 30 new servers the MDC
> bought for ci.guix.gnu.org, but I’d be happier if we figured out what’s
> wrong with bayfront instead of centralizing all services in Berlin…

I'm definitely not saying anything is wrong with Bayfront, just that it
would be good to maybe check what the expectations are with the services
it hosts. Given that the remote access only allows power cycling the
machine, if that doesn't work, there's the potential for downtime for
those services to last until someone physically goes to the machine to
fix it, if SSH access can't be restored by turning it off and on again.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Using the Hetzner Cloud

[Ludovic Courtès]

Hi Christopher,

Christopher Baines <mail@cbaines.net> skribis:

> I've been using Hetzner for hosting data.guix.gnu.org for at least the
> last few months now. I've been paying for it so far, but ideally it
> wouldn't depend on me personally paying for it in the long term.
>
> Coupled with this, I think there are some services (hpc.guix.info, ...)
> it would be nice to move off bayfront (a machine in Bordeaux) to
> somewhere that's more reliable/remotely accessible, mostly to avoid the
> risk of me breaking them again when attempting to reconfigure bayfront.
>
> Maybe Guix Europe (a legal entity in Europe) could have a Hetzner
> account that could be used at the discretion of those involved in Guix
> Europe? I think Hetzner support billing by SEPA direct debit, which
> might work, otherwise I can probably pay the bill and then be
> reimbursed.

Yes, we should get that done.

Alternately, if Guix Europe cannot do it, we can at least get you
reimbursed via our funds at the Free Software Foundation.

Andreas, Manolis: thoughts?

Ludo’.