News from Cuirass

News from Cuirass

[Ludovic Courtès]

Hello Guix!

A couple of months ago, I found myself working on hot fixes for Cuirass,
whose repository had been gathering dust for months.  One thing leads to
another, and I’ve worked on a few improvements recently:

  • Guile-Squee (interface to PostgreSQL) now cooperates with Fibers,
    meaning that ‘exec-query’ is suspendable.

  • Thanks to advice from Chris Baines, Cuirass now uses a database
    connection pool instead of the database service thread it used to
    have.

  • ‘cuirass remote-server’ and ‘cuirass remote-worker’, the processes
    in charge of distributing builds remotely and that communicate over
    ZeroMQ, are now single fiberized processes (this was made possible
    by making the ‘receive-message’ procedure, built upon
    Guile-Simple-ZMQ, cooperative).  These commands used to spawn a
    bunch of processes and threads.

  • Assorted improvements to the web interface and to logging.

I’m regularly deploying the latest revision at
<https://guix.bordeaux.inria.fr> using the Cuirass channel (I’m
purposefully postponing an update of the ‘cuirass’ package until I’ve
acquired more experience and confidence.)  If you’re running an instance
of Cuirass and feeling adventurous, you can try it as well:

  (cons* (channel
          (name 'cuirass)
          (url "https://git.savannah.gnu.org/git/guix/guix-cuirass.git")
          (introduction
           (make-channel-introduction
            "c75620777c33273fcd14261660288ec1b2dc8123"
            (openpgp-fingerprint
             "3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5"))))
         %default-channels)

With these architectural changes, we should be able to make more visible
improvements such as adding web hooks (so a Git repo can trigger a new
evaluation), replacing polling with notifications in many cases, adding
a Build Coordinator backend (convergence!), etc.  See
<https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/tree/TODO>.

Contributions welcome!  :-)

Ludo’.

Re: News from Cuirass

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Ludovic,

On Sun, Jul 16, 2023 at 6:53 AM Ludovic Courtès <ludo@gnu.org> wrote:
>
>     Cuirass now uses a database
>     connection pool

That is super exciting! In another implementation the pooling code
comes with the database interface [1] which makes it available more
broadly. Could the pooling code, which presumably lives here, [2]
perhaps become part of Guile-Squee?

Also, Kudos on your dive into ZeroMQ! I hope you find the experience
as rewarding as I did.

Kind regards & thanks,
Felix

[1] https://node-postgres.com/features/pooling
[2] https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/tree/src/cuirass/database.scm#n269

Re: News from Cuirass

[Ludovic Courtès]

Hello!

Felix Lechner <felix.lechner@lease-up.com> skribis:

> On Sun, Jul 16, 2023 at 6:53 AM Ludovic Courtès <ludo@gnu.org> wrote:
>>
>>     Cuirass now uses a database
>>     connection pool
>
> That is super exciting! In another implementation the pooling code
> comes with the database interface [1] which makes it available more
> broadly. Could the pooling code, which presumably lives here, [2]
> perhaps become part of Guile-Squee?

The pool implementation is here:

  https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/tree/src/cuirass/utils.scm?id=c4743b54720e86b0e0b0295fb6d33977e4293644#n96

Currently Guile-Squee does not depend on Fibers, so I don’t think we can
move the pool there, or at least not without changing that (but I think
it’s good that one can use Guile-Squee without Fibers).

Thanks,
Ludo’.