unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
From: "Clément Lassieur" <clement@lassieur.org>
To: 32190@debbugs.gnu.org
Subject: bug#32190: Cuirass doesn't check if two subsequent jobs yield the same derivation
Date: Wed, 18 Jul 2018 00:32:02 +0200	[thread overview]
Message-ID: <87efg1ijdp.fsf@lassieur.org> (raw)
In-Reply-To: <87k1ptirr0.fsf@lassieur.org>

Consider the following table:

--8<---------------cut here---------------start------------->8---
CREATE TABLE Derivations (
  derivation    TEXT NOT NULL,
  evaluation    INTEGER NOT NULL,
  job_name      TEXT NOT NULL,
  system        TEXT NOT NULL,
  nix_name      TEXT NOT NULL,
  PRIMARY KEY (derivation, evaluation),
  FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
);
--8<---------------cut here---------------end--------------->8---

And the following code:

--8<---------------cut here---------------start------------->8---
(define (db-add-derivation db job)
  "Store a derivation result in database DB and return its ID."
  (catch 'sqlite-error
    (lambda ()
      (sqlite-exec db "\
INSERT INTO Derivations (derivation, job_name, system, nix_name, evaluation)\
  VALUES ("
                   (assq-ref job #:derivation) ", "
                   (assq-ref job #:job-name) ", "
                   (assq-ref job #:system) ", "
                   (assq-ref job #:nix-name) ", "
                   (assq-ref job #:eval-id) ");")
      (last-insert-rowid db))
    (lambda (key who code message . rest)
      ;; If we get a unique-constraint-failed error, that means we have
      ;; already inserted the same (derivation,eval-id) tuple.  That happens
      ;; when several jobs produce the same derivation, and we can ignore it.
      (if (= code SQLITE_CONSTRAINT_PRIMARYKEY)
          (sqlite-exec db "SELECT * FROM Derivations WHERE derivation="
                       (assq-ref job #:derivation) ";")
          (apply throw key who code rest)))))
--8<---------------cut here---------------end--------------->8---

I think the above constraint can't happen because by definition a new
job (for the same job_name) is produced at each evaluation.  So eval-id
will be incremented every time.

Also, the docs (and a comment in schema.sql) says:

    Builds are not in a one to one relationship with derivations in
    order to keep track of non deterministic compilations.

But I think it doesn't make sense, because Guix won't try to build twice
the same thing unless '--check' is used (which obviously isn't the
case).

So not only we have a huge Derivations table full of identical items,
but we also ask Guix to build them and we store the results in the
Builds table...

Maybe the solution is to replace the (derivation, evaluation) primary
key with (derivation), and only build the newly added derivations.
WDYT?

Clément

  reply	other threads:[~2018-07-17 22:33 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAMSS15C0nNEPqQKjRt9=-JKFvrGZsKBtMOxju65p_y88EzOZgg@mail.gmail.com>
     [not found] ` <87vac3twbe.fsf@gnu.org>
     [not found]   ` <87o9hog2ye.fsf@elephly.net>
     [not found]     ` <CAMSS15CQwypdFafPG1ii-CgMpxYxLtcbx8hHTvDvyBS+6xNSxA@mail.gmail.com>
     [not found]       ` <87d0xyn9zs.fsf@elephly.net>
     [not found]         ` <CAMSS15CGdzb8=-Oz6z6bmeWjrnT85PL+q7DLUFw-9E4_d4Y6pw@mail.gmail.com>
     [not found]           ` <87d0xswvls.fsf@elephly.net>
     [not found]             ` <CAMSS15CDGat-pFjiz2vrkvb14qWnY4rbCW-d8KSzY6MO7WzT_g@mail.gmail.com>
     [not found]               ` <87r2m4ntk4.fsf@mdc-berlin.de>
     [not found]                 ` <CAMSS15D42gCV-UneBBJWngdfi-jG4JfWWh6NWhWMiet0Y=bUsg@mail.gmail.com>
     [not found]                   ` <87tvqxy4i9.fsf@elephly.net>
     [not found]                     ` <CAMSS15DThnLO+YEVaBmJ9ozMeu4mO1rHAdXHgZ8K+Csu40pORQ@mail.gmail.com>
     [not found]                       ` <87in78hxo2.fsf@elephly.net>
2018-07-17 19:31                         ` GSoC: Adding a web interface similar to the Hydra web interface Clément Lassieur
2018-07-17 22:32                           ` Clément Lassieur [this message]
2018-07-24 10:05                             ` bug#32190: Cuirass doesn't check if two subsequent jobs yield the same derivation Ludovic Courtès
2018-08-04 16:03                           ` bug#32190: [PATCH] database: Merge Derivations into Builds table Clément Lassieur
2018-08-04 16:09                             ` Clément Lassieur
2018-08-08 12:13                             ` Clément Lassieur
2018-08-14 16:57                             ` Clément Lassieur
2018-08-14 19:04                             ` Ricardo Wurmus
2018-08-15 18:57                               ` Clément Lassieur
2018-08-16 21:00                               ` Clément Lassieur

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87efg1ijdp.fsf@lassieur.org \
    --to=clement@lassieur.org \
    --cc=32190@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).