Re: Sustainable funding and maintenance for our infrastructure

[Enrico Schwass <ennoausberlin@me.com>]

Hi Guix

If you are looking for host services
I have good experience with netcup.de. They also have ARM machines.
My 3 machines have a great uptime

Bye
Enno


> Am 2024/07/08 um 18:28 schrieb Efraim Flashner <efraim@flashner.co.il>:
> 
> On Tue, Jul 02, 2024 at 04:24:06PM +0200, Ludovic Courtès wrote:
>> Hello Guix!
>> 
>> We (Andreas, Chris, Ricardo, Romain, and myself) were having a
>> discussion about what it would take to set up a build farm similar to
>> what’s behind ci.guix: roughly 30 x86_64 servers, with 32-core/64-thread
>> CPUs and 128 GiB of RAM.  The reason for this discussion is that we were
>> thinking that we should not take our existing build farms for granted
>> and be prepared for the future.
>> 
>> The various options and back-of-the-envelope estimates we came up with
>> are as follows:
>> 
>>  1. Buying and hosting hardware:
>>      250k€ for hardware
>>      3k€/month (36k€/year)
>> 
>>  2. Renting machines (e.g., on Hetzner):
>>      6k€/month (72k€/year)
>> 
>>  3. Sponsored:
>>      get hardware and/or hosting sponsored (by academic institutions or
>>      companies).
>> 
>> Option #1 gives us “full control”, the downside being that it’s a lot of
>> work and a real burden (get crowdfunding for the initial funding, later
>> on to sustain funding to cover hosting, ensure Guix Foundation is up to
>> the task of managing the assets, and of course to take care of the
>> machines for their entire lifecycle).
>> 
>> Option #2 gives us less control (we don’t know exactly what hardware is
>> being used and have to trust the company hosting the machines).  The
>> upside is that it’s much less work over time (the company is responsible
>> for upgrading hardware) and less work initially (no need to raise as
>> much money to buy hardware).
>> 
>> Option #3 potentially gives less control (depending on the project’s
>> relation with the hosting organization) and makes the project dependent
>> on the sponsor and/or person(s) in touch with them.  On the upside, it
>> could significantly reduce costs (potentially to 0€).
>> 
>> 
>> This is an important topic for the project, one we should plan for:
>> socially, financially, technically.  This takes time, which is why
>> preparation is needed.
>> 
>> What do people think?
>> 
>> Ludo’ & co.
> 
> Looking at Hetzner, they have an option to rent a dedicated ARM server
> with 80 cores/threads with 256GB of RAM and 2x3.84 TB NVMe drives for
> under €300/month and a €94 setup charge. Correct me if I"m wrong, but
> that one box is ~20x our current active aarch64/armv7 capacity.
> 
> Also looking at our current infrastructure at MDC, part of the reason we
> have so many x86_64 machines is because that's what was bought with the
> donated money, not because we actually needed quite that many, so some
> of the numbers might be higher than we actually need.
> 
> -- 
> Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
> GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted
> <signature.asc>


> Am 2024/07/08 um 18:28 schrieb Efraim Flashner <efraim@flashner.co.il>:
> 
> On Tue, Jul 02, 2024 at 04:24:06PM +0200, Ludovic Courtès wrote:
>> Hello Guix!
>> 
>> We (Andreas, Chris, Ricardo, Romain, and myself) were having a
>> discussion about what it would take to set up a build farm similar to
>> what’s behind ci.guix: roughly 30 x86_64 servers, with 32-core/64-thread
>> CPUs and 128 GiB of RAM.  The reason for this discussion is that we were
>> thinking that we should not take our existing build farms for granted
>> and be prepared for the future.
>> 
>> The various options and back-of-the-envelope estimates we came up with
>> are as follows:
>> 
>>  1. Buying and hosting hardware:
>>      250k€ for hardware
>>      3k€/month (36k€/year)
>> 
>>  2. Renting machines (e.g., on Hetzner):
>>      6k€/month (72k€/year)
>> 
>>  3. Sponsored:
>>      get hardware and/or hosting sponsored (by academic institutions or
>>      companies).
>> 
>> Option #1 gives us “full control”, the downside being that it’s a lot of
>> work and a real burden (get crowdfunding for the initial funding, later
>> on to sustain funding to cover hosting, ensure Guix Foundation is up to
>> the task of managing the assets, and of course to take care of the
>> machines for their entire lifecycle).
>> 
>> Option #2 gives us less control (we don’t know exactly what hardware is
>> being used and have to trust the company hosting the machines).  The
>> upside is that it’s much less work over time (the company is responsible
>> for upgrading hardware) and less work initially (no need to raise as
>> much money to buy hardware).
>> 
>> Option #3 potentially gives less control (depending on the project’s
>> relation with the hosting organization) and makes the project dependent
>> on the sponsor and/or person(s) in touch with them.  On the upside, it
>> could significantly reduce costs (potentially to 0€).
>> 
>> 
>> This is an important topic for the project, one we should plan for:
>> socially, financially, technically.  This takes time, which is why
>> preparation is needed.
>> 
>> What do people think?
>> 
>> Ludo’ & co.
> 
> Looking at Hetzner, they have an option to rent a dedicated ARM server
> with 80 cores/threads with 256GB of RAM and 2x3.84 TB NVMe drives for
> under €300/month and a €94 setup charge. Correct me if I"m wrong, but
> that one box is ~20x our current active aarch64/armv7 capacity.
> 
> Also looking at our current infrastructure at MDC, part of the reason we
> have so many x86_64 machines is because that's what was bought with the
> donated money, not because we actually needed quite that many, so some
> of the numbers might be higher than we actually need.
> 
> --
> Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
> GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted
> <signature.asc>