unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
From: Mathieu Othacehe <othacehe@gnu.org>
To: 43588-done@debbugs.gnu.org
Subject: bug#43588: cuirass: Slow SQL queries.
Date: Mon, 28 Sep 2020 18:02:21 +0200	[thread overview]
Message-ID: <87eemldh4i.fsf@gnu.org> (raw)
In-Reply-To: <878scz75u0.fsf@gnu.org> (Mathieu Othacehe's message of "Thu, 24 Sep 2020 13:49:11 +0200")


Hello,

> SELECT E.id, E.status, E.timestamp, E.checkouttime, E.evaltime, B.total, B.succeeded, B.failed, B.scheduled FROM (SELECT id, status, timestamp, checkouttime, evaltime FROM Evaluations WHERE (id='8255' )) E LEFT JOIN (SELECT rowid, evaluation, SUM(status=0) as succeeded, SUM(status>0) as failed, SUM(status<0) as scheduled, SUM(status>-100) as total FROM Builds GROUP BY evaluation) B ON B.evaluation=E.id ORDER BY E.id ASC 86.23

Turns out those queries are really slow because they are doing full
scans of the Builds table. Using "EXPLAIN QUERY PLAN" shows that this is
caused by the "GROUP BY" in the "B" sub-query.

Running the above query takes around 3 seconds right now. I think that
it took 86.23 seconds because the WAL file contained a lot of
temporary data, slowing down the whole query. I ran a "PRAGMA
wal_checkpoint(FULL)" as those queries were started to slow down and
observed an immediate improvement.

Anyway, the real problem here is just highlighted by the WAL file
usage. Those queries can really go faster.

I rewrote them and added a few indexes with
0ffcb80ebbaa2b177f03548035a2ef21ae7ac41d. They now take less than 1ms,
which should very much improve the web browsing experience as well as
decrease the database worker starvation.

Thanks,

Mathieu
-- 
https://othacehe.org




      reply	other threads:[~2020-09-28 16:04 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-24 11:49 bug#43588: cuirass: Slow SQL queries Mathieu Othacehe
2020-09-28 16:02 ` Mathieu Othacehe [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87eemldh4i.fsf@gnu.org \
    --to=othacehe@gnu.org \
    --cc=43588-done@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).