From: Mathieu Othacehe <othacehe@gnu.org>
To: guix-devel@gnu.org
Subject: CI status
Date: Wed, 15 Dec 2021 17:15:08 +0100 [thread overview]
Message-ID: <87zgp1j14j.fsf@gnu.org> (raw)
Hello,
You must have noticed that the CI is currently struggling a bit. Here is
a small recap of the situation.
* The IO operations on Berlin are mysteriously slow. Removing files from
/gnu/store/trash is taking ages. This is reported here:
https://issues.guix.gnu.org/51787.
We have to kill the garbage collect frequently to keep things
going. The bad side is obviously that we can't do that forever, as we
only have 9.3T and decreasing, while we aim to stay at 10T available.
* The PostgreSQL database behind ci.guix.gnu.org also became super slow
and I decided to drop it. I don't know if there's a connection with
the above point. I'm missing the appropriate tools/knowledge to
monitor the IO & file-system performances.
* The php package isn't building anymore, reported here:
https://issues.guix.gnu.org/52513. This means that we cannot
reconfigure zabbix. I removed it from the berlin configuration
temporarily.
* The cuirass-remote-server Avahi service is no longer visible when
running "avahi-browse -a". I strongly suspect that this is related to
the static-networking update, even if I don't have a proof for
now. This means that the remote-workers using Avahi for discovering
(hydra-guix-*) machines can no longer connect. The
ci.guix.gnu.org/workers list is thus quite empty.
* Facing those problems, I tried to rollback to a previous system
generation, but this is bringing even more issues, as for instance the
older Cuirass package, is struggling with the new database structure and
other niceties. I think out best course of action is to stick to
master and fix the above problems.
Thanks,
Mathieu
next reply other threads:[~2021-12-15 16:15 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-15 16:15 Mathieu Othacehe [this message]
2021-12-15 17:26 ` CI status Leo Famulari
2021-12-15 19:38 ` Mathieu Othacehe
2021-12-15 19:43 ` Leo Famulari
2021-12-15 20:36 ` Ricardo Wurmus
2021-12-20 2:16 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87zgp1j14j.fsf@gnu.org \
--to=othacehe@gnu.org \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.