[bug#74046] [PATCH] doc: Add "Contributing to Guix's infrastructure".

[bug#74046] [PATCH] doc: Add "Contributing to Guix's infrastructure".

[Ekaitz Zarraga]

Use the "Call for contribution to the Guix infrastructure" by Ludovic
Courtès to create a section in the documentation that describes how to
contribute to the infrastructure.

https://lists.gnu.org/archive/html/guix-devel/2024-05/msg00183.html

* doc/contributing.texi (Contributing to Guix's infrastructure): New
  section.

Change-Id: I3f3a99ad884110cc8323789e8c14bec1f7327e97
---
 doc/contributing.texi | 187 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index acdc303be6..1510e07ddc 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -37,6 +37,7 @@ Contributing
 * Deprecation Policy::          Commitments and tools for deprecation.
 * Writing Documentation::       Improving documentation in GNU Guix.
 * Translating Guix::            Make Guix speak your native language.
+* Contributing to Guix's infrastructure::  Make Guix ecosystem work better.
 @end menu
 
 @node Requirements
@@ -3594,3 +3595,189 @@ Translating Guix
       be updated accordingly (see @file{website/i18n-howto.txt} for more
       information on the process).
 @end itemize
+
+
+@cindex infrastructure
+@node Contributing to Guix's infrastructure
+@section Contributing to Guix's infrastructure
+
+Since its inception, the Guix project has always valued its autonomy, and that
+reflects in its infrastructure: our servers run Guix System and exclusively
+free software, none of them is hosted by one of these transnational companies,
+and they're administered by volunteers.
+
+Of course this comes at a cost and this is why we're sending this call for
+contributions.  Our hope is to make infrastructure-related activity more
+legible so that maybe you can picture yourself helping in one of these areas.
+
+
+@menu
+* Coding::
+* System administration::
+* Day-to-day system administration::
+* On-site intervention::
+* Hosting::
+* Administrative tasks::
+@end menu
+
+@node Coding
+@subsection Coding
+
+Guix runs many Guix-specific services; this is all lovely Scheme code but it
+tends to receive less attention than Guix itself:
+
+@itemize
+@item Build Farm Front-End: @url{https://git.cbaines.net/guix/bffe}
+@item Cuirass: @url{https://guix.gnu.org/cuirass/}
+@item Goggles (IRC logger):
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/goggles.scm}
+@item Guix Build Coordinator:
+@url{https://git.savannah.gnu.org/cgit/guix/build-coordinator.git/}
+@item Guix Data Service:
+@url{https://git.savannah.gnu.org/git/guix/data-service.git/}
+@item Guix Packages Website:
+@url{https://codeberg.org/luis-felipe/guix-packages-website.git}
+@item mumi: @url{https://git.savannah.gnu.org/cgit/guix/mumi.git/}
+@item nar-herder: @url{https://git.savannah.gnu.org/cgit/guix/nar-herder.git/}
+@item QA Frontpage: @url{https://git.savannah.gnu.org/git/guix/qa-frontpage.git}
+@end itemize
+
+There is no time constraint on this coding activity: any improvement is
+welcome, whenever it comes.  Most of these code bases are relatively small,
+which should make it easier to get started.
+
+Prerequisites: Familiarity with Guile, HTTP, and databases.
+
+If you wish to get started, check out the README of the project of your choice
+and get in touch with guix-devel and the primary developer(s) of the tool as
+per @code{git shortlog -s | sort -k1 -n}.
+
+@node System administration
+@subsection System administration
+
+Guix System configuration for all our systems is held in this repository:
+
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/}
+
+The two front-ends are @file{berlin.scm} (the machine behind ci.guix.gnu.org)
+and @file{bayfront.scm} (the machine behind bordeaux.guix.gnu.org,
+guix.gnu.org, hpc.guix.info, qa.guix.gnu.org, and more).  Both connect to a
+number of build machines and helpers.
+
+Without even having SSH access to the machine, you can help by posting patches
+to improve the configuration (you can test it with @code{guix system vm}).
+Here are ways you can help:
+
+@itemize
+@item
+Improve infra monitoring: set up a dashboard to monitor all the infrastructure,
+and an out-of-band channel to communicate about downtime.
+
+@item
+Implement web site redundancy: guix.gnu.org should be backed by several
+machines on different sites.  Get in touch with us and/or send a patch!
+
+@item
+Implement substitute redundancy: likewise, bordeaux.guix.gnu.org and
+ci.guix.gnu.org should be backed by several head nodes.
+
+@item
+Improve backup: there's currently ad-hoc backup of selected pieces over rsync
+between the two head nodes; we can improve on that, for example with a
+dedicated backup site and proper testing of recoverability.
+
+@item
+Support mirroring: We'd like to make it easy for others to mirror substitutes
+from ci.guix and bordeaux.guix, perhaps by offering public rsync access.
+
+@item
+Optimize our web services: Monitor the performance of our services and tweak
+nginx config or whatever it takes to improve it.
+
+There is no time constraint on this activity: any improvement is welcome,
+whenever you can work on it.
+
+Prerequisite: Familiarity with Guix System administration and ideally with the
+infrastructure handbook:
+
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/infra-handbook.org}
+
+@end itemize
+
+@node Day-to-day system administration
+@subsection Day-to-day system administration
+
+We're also looking for people who'd be willing to have SSH access to some of
+the infrastructure to help with day-to-day maintenance: restarting a build,
+restarting the occasional service that has gone wild (that can happen),
+reconfiguring/upgrading a machine, rebooting, etc.
+
+This day-to-day activity requires you to be available some of the time (during
+office hours or not, during the week-end or not), whenever is convenient for
+you, so you can react to issues reported on IRC, on the mailing list, or
+elsewhere, and synchronize with other sysadmins.
+
+Prerequisite: Being a “known” member of the community, familiarity with Guix
+System administration, with some of the services/web sites being run, and with
+the infrastructure handbook:
+
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/infra-handbook.org}
+
+@node On-site intervention
+@subsection On-site intervention
+
+The first front-end is currently generously hosted by the Max Delbrück Center
+(MDC), a research institute in Berlin, Germany. Only authorized personnel can
+physically access it.
+
+The second one, bordeaux.guix.gnu.org, is hosted in Bordeaux, France, in a
+professional data center shared with non-profit ISP Aquilenet.  If you live in
+the region of Bordeaux and would like to help out when we need to go on-site,
+please make yourself known by emailing @email{guix-sysadmin@@gnu.org}.
+
+On-site interventions are rare, but they're usually in response to an
+emergency.
+
+@node Hosting
+@subsection Hosting
+
+We're looking for people who can host machines and help out whenever
+physical access is needed.  More specifically:
+
+@itemize
+@item
+We need hosting of “small” machines such as single-board computers (AArch64,
+RISC-V) for use as build machines.
+
+@item
+We need hosting for front-ends and x86_64 build machines in a data center where
+they can be racked and where, ideally, several local Guix sysadmins can
+physically access them.
+@end itemize
+
+The machines should be accessible over Wireguard VPN most of the
+time, so longer power or network interruptions should be the
+exception.
+
+Prerequisites: Familiarity with installing and remotely administering Guix
+System.
+
+@node Administrative tasks
+@subsection Administrative tasks
+
+The infra remains up and running thanks to crucial administrative tasks, which
+includes:
+
+@itemize
+@item
+Selecting and purchasing hardware, for example build machines.
+@item
+Renewing domain names.
+
+@item
+Securing funding, in particular via the Guix Foundation:
+@url{https://foundation.guix.info}
+@end itemize
+
+Prerequisites: Familiarity with hardware, and/or DNS registrars,
+and/or sponsorship, and/or crowdfunding.

base-commit: 091131af64fd4e4e925fff829fa19097cfcdfcc5
-- 
2.46.0

[bug#74046] [PATCH] doc: Add "Contributing to Guix's infrastructure".

[Greg Hogan]

On Sun, Oct 27, 2024 at 8:46 AM Ekaitz Zarraga <ekaitz@elenq.tech> wrote:
>
> +Since its inception, the Guix project has always valued its autonomy, and that
> +reflects in its infrastructure: our servers run Guix System and exclusively
> +free software, none of them is hosted by one of these transnational companies,
> +and they're administered by volunteers.

Would it be better to remove the negative reference to "transnational
companies" and replace it with a positive description of the current
and desired hosting sites? Or remove that snippet altogether? I think
this is saying we don't like "clouds", and if the reason is trusted
hardware then we can simply state that and note the requisite free
software bootloader.

Greg

[bug#74046] [PATCH] doc: Add "Contributing to Guix's infrastructure".

[Nicolas Graves via Guix-patches via]

I have no particular knowledge about distributed computing other than a
past use of BOINC, but I wonder if some contribution in that form would
be useful in a near future in Guix.  I have a beefy machine and would be
happy to lend ~10 cores for 10h/day to building Guix binaries.  Not that
much, but I like the idea, and I think it also helps to create another
contribution option and might participate to create a stronger
community. (By the way, same thing for peer-sharing build results, with
the same problematics).

Has there already been talks about that?  I guess the hard part is to
guarantee security / untouched contributed binaries but I guess guys
from distributed computing also have the same issues, so maybe there's a
way to guarantee that? 

-- 
Best regards,
Nicolas Graves

[bug#74046] [PATCH v2] doc: Add "Contributing to Guix's infrastructure".

[Ekaitz Zarraga]

Use the "Call for contribution to the Guix infrastructure" by Ludovic
Courtès to create a section in the documentation that describes how to
contribute to the infrastructure.

https://lists.gnu.org/archive/html/guix-devel/2024-05/msg00183.html

* doc/contributing.texi (Contributing to Guix's infrastructure): New
  section.

Change-Id: I3f3a99ad884110cc8323789e8c14bec1f7327e97
---
 doc/contributing.texi | 186 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 186 insertions(+)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index acdc303be6..1fb6ce2e1e 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -37,6 +37,7 @@ Contributing
 * Deprecation Policy::          Commitments and tools for deprecation.
 * Writing Documentation::       Improving documentation in GNU Guix.
 * Translating Guix::            Make Guix speak your native language.
+* Contributing to Guix's infrastructure::  Make Guix ecosystem work better.
 @end menu
 
 @node Requirements
@@ -3594,3 +3595,188 @@ Translating Guix
       be updated accordingly (see @file{website/i18n-howto.txt} for more
       information on the process).
 @end itemize
+
+
+@cindex infrastructure
+@node Contributing to Guix's infrastructure
+@section Contributing to Guix's infrastructure
+
+Since its inception, the Guix project has always valued its autonomy, and that
+reflects in its infrastructure: our servers run Guix System and exclusively
+free software, and are administered by volunteers.
+
+Of course this comes at a cost and this is why we need contributions.  Our hope
+is to make infrastructure-related activity more legible so that maybe you can
+picture yourself helping in one of these areas.
+
+
+@menu
+* Coding::
+* System administration::
+* Day-to-day system administration::
+* On-site intervention::
+* Hosting::
+* Administrative tasks::
+@end menu
+
+@node Coding
+@subsection Coding
+
+Guix runs many Guix-specific services; this is all lovely Scheme code but it
+tends to receive less attention than Guix itself:
+
+@itemize
+@item Build Farm Front-End: @url{https://git.cbaines.net/guix/bffe}
+@item Cuirass: @url{https://guix.gnu.org/cuirass/}
+@item Goggles (IRC logger):
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/goggles.scm}
+@item Guix Build Coordinator:
+@url{https://git.savannah.gnu.org/cgit/guix/build-coordinator.git/}
+@item Guix Data Service:
+@url{https://git.savannah.gnu.org/git/guix/data-service.git/}
+@item Guix Packages Website:
+@url{https://codeberg.org/luis-felipe/guix-packages-website.git}
+@item mumi: @url{https://git.savannah.gnu.org/cgit/guix/mumi.git/}
+@item nar-herder: @url{https://git.savannah.gnu.org/cgit/guix/nar-herder.git/}
+@item QA Frontpage: @url{https://git.savannah.gnu.org/git/guix/qa-frontpage.git}
+@end itemize
+
+There is no time constraint on this coding activity: any improvement is
+welcome, whenever it comes.  Most of these code bases are relatively small,
+which should make it easier to get started.
+
+Prerequisites: Familiarity with Guile, HTTP, and databases.
+
+If you wish to get started, check out the README of the project of your choice
+and get in touch with guix-devel and the primary developer(s) of the tool as
+per @code{git shortlog -s | sort -k1 -n}.
+
+@node System administration
+@subsection System administration
+
+Guix System configuration for all our systems is held in this repository:
+
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/}
+
+The two front-ends are @file{berlin.scm} (the machine behind ci.guix.gnu.org)
+and @file{bayfront.scm} (the machine behind bordeaux.guix.gnu.org,
+guix.gnu.org, hpc.guix.info, qa.guix.gnu.org, and more).  Both connect to a
+number of build machines and helpers.
+
+Without even having SSH access to the machine, you can help by posting patches
+to improve the configuration (you can test it with @code{guix system vm}).
+Here are ways you can help:
+
+@itemize
+@item
+Improve infra monitoring: set up a dashboard to monitor all the infrastructure,
+and an out-of-band channel to communicate about downtime.
+
+@item
+Implement web site redundancy: guix.gnu.org should be backed by several
+machines on different sites.  Get in touch with us and/or send a patch!
+
+@item
+Implement substitute redundancy: likewise, bordeaux.guix.gnu.org and
+ci.guix.gnu.org should be backed by several head nodes.
+
+@item
+Improve backup: there's currently ad-hoc backup of selected pieces over rsync
+between the two head nodes; we can improve on that, for example with a
+dedicated backup site and proper testing of recoverability.
+
+@item
+Support mirroring: We'd like to make it easy for others to mirror substitutes
+from ci.guix and bordeaux.guix, perhaps by offering public rsync access.
+
+@item
+Optimize our web services: Monitor the performance of our services and tweak
+nginx config or whatever it takes to improve it.
+
+There is no time constraint on this activity: any improvement is welcome,
+whenever you can work on it.
+
+Prerequisite: Familiarity with Guix System administration and ideally with the
+infrastructure handbook:
+
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/infra-handbook.org}
+
+@end itemize
+
+@node Day-to-day system administration
+@subsection Day-to-day system administration
+
+We're also looking for people who'd be willing to have SSH access to some of
+the infrastructure to help with day-to-day maintenance: restarting a build,
+restarting the occasional service that has gone wild (that can happen),
+reconfiguring/upgrading a machine, rebooting, etc.
+
+This day-to-day activity requires you to be available some of the time (during
+office hours or not, during the week-end or not), whenever is convenient for
+you, so you can react to issues reported on IRC, on the mailing list, or
+elsewhere, and synchronize with other sysadmins.
+
+Prerequisite: Being a “known” member of the community, familiarity with Guix
+System administration, with some of the services/web sites being run, and with
+the infrastructure handbook:
+
+@url{https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/infra-handbook.org}
+
+@node On-site intervention
+@subsection On-site intervention
+
+The first front-end is currently generously hosted by the Max Delbrück Center
+(MDC), a research institute in Berlin, Germany. Only authorized personnel can
+physically access it.
+
+The second one, bordeaux.guix.gnu.org, is hosted in Bordeaux, France, in a
+professional data center shared with non-profit ISP Aquilenet.  If you live in
+the region of Bordeaux and would like to help out when we need to go on-site,
+please make yourself known by emailing @email{guix-sysadmin@@gnu.org}.
+
+On-site interventions are rare, but they're usually in response to an
+emergency.
+
+@node Hosting
+@subsection Hosting
+
+We're looking for people who can host machines and help out whenever
+physical access is needed.  More specifically:
+
+@itemize
+@item
+We need hosting of “small” machines such as single-board computers (AArch64,
+RISC-V) for use as build machines.
+
+@item
+We need hosting for front-ends and x86_64 build machines in a data center where
+they can be racked and where, ideally, several local Guix sysadmins can
+physically access them.
+@end itemize
+
+The machines should be accessible over Wireguard VPN most of the
+time, so longer power or network interruptions should be the
+exception.
+
+Prerequisites: Familiarity with installing and remotely administering Guix
+System.
+
+@node Administrative tasks
+@subsection Administrative tasks
+
+The infra remains up and running thanks to crucial administrative tasks, which
+includes:
+
+@itemize
+@item
+Selecting and purchasing hardware, for example build machines.
+@item
+Renewing domain names.
+
+@item
+Securing funding, in particular via the Guix Foundation:
+@url{https://foundation.guix.info}
+@end itemize
+
+Prerequisites: Familiarity with hardware, and/or DNS registrars,
+and/or sponsorship, and/or crowdfunding.

base-commit: 091131af64fd4e4e925fff829fa19097cfcdfcc5
-- 
2.46.0