unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: Mathieu Othacehe <othacehe@gnu.org>
Cc: 46585@debbugs.gnu.org
Subject: [bug#46585] ci: Remove hydra support.
Date: Mon, 22 Feb 2021 10:59:11 +0100	[thread overview]
Message-ID: <87pn0sctjk.fsf@gnu.org> (raw)
In-Reply-To: <87h7mbgk6e.fsf@gnu.org> (Mathieu Othacehe's message of "Wed, 17 Feb 2021 09:42:01 +0100")

Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

> This removes hydra support to use Cuirass as the only continuous
> integration system.

Yay!  That part could have been a separate commit, for clarity, but
that’s okay.

> It also simplifies the evaluation process. Here's how it's working
> now:
>
> * Cuirass fetches new commits and calls its "evaluate" process.
>
> * The "evaluate" process calls the "cuirass-jobs" procedure in the
> newly checkouted Guix "build-aux/cuirass/gnu-system.scm" file.
>
> * The "hydra-jobs" procedure in "build-aux/hydra/gnu-system.scm" file
>   starts the evaluation of "hydra-jobs" of (gnu ci) module in an
>   inferior using the new commit.
>
> * This procedure returns the list of all the package derivations at that
>   very commit under Hydra job format.
>
> * This list is converted to the Cuirass job format and written on the
>   stdout port.
>
> * The main Cuirass process reads the "evaluate" output using a pipe, and
>   registers the derivation that needs to be built in database.
>
> This is quite complex and it requires to pass around a huge list of
> jobs, consuming a lot of memory.
>
> Here's the simplified method:
>
> * The first two steps are identical.
>
> * The "cuirass-jobs" procedure starts the evaluation of "cuirass-jobs"
>   of (gnu ci) module in an inferior using the new commit. This procedure
>   is passed a registration callback that directly registers the given
>   jobs in database. It doesn't return anything.
>
> As the "register" procedure is a part of Cuirass, the inversion on
> control caused by the inferior is problematic. I had to proxy the
> registration requests from the inferior to the evaluation process using
> a named pipe.

OK.  Another option, to avoid the callback, would have been to write
jobs to stdout as before, but to write them one by one instead of
building the entire list in memory.  Perhaps that would be slightly
simpler than the callback + named pipe?

> Other than that, the process seems now less obfuscated.

Yeah, thumbs up.  Do you know how this affects memory consumption?

Thanks!

Ludo’.




  reply	other threads:[~2021-02-22 10:00 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-17  8:42 [bug#46585] ci: Remove hydra support Mathieu Othacehe
2021-02-22  9:59 ` Ludovic Courtès [this message]
2021-03-26  9:52   ` bug#46585: " Mathieu Othacehe
2021-03-24  3:25 ` [bug#46585] " zimoun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87pn0sctjk.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=46585@debbugs.gnu.org \
    --cc=othacehe@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).