From: Ricardo Wurmus <rekado@elephly.net>
To: "Clément Lassieur" <clement@lassieur.org>
Cc: 32190@debbugs.gnu.org
Subject: bug#32190: [PATCH] database: Merge Derivations into Builds table.
Date: Tue, 14 Aug 2018 21:04:23 +0200 [thread overview]
Message-ID: <87h8jwvkg8.fsf@elephly.net> (raw)
In-Reply-To: <20180804160303.20451-1-clement@lassieur.org>
Hi Clément,
> Fixes <https://bugs.gnu.org/32190>.
Woo! Thank you for this patch.
> * src/cuirass/base.scm (evaluate): Don't add jobs to the Derivations table.
I see that you’ve mentioned your changes to “build-packages” in a later
email.
I have two general questions about this: why was the change from “id” to
“rowid” necessary? And: could you please also update the documentation
so that is reflects the changes?
> diff --git a/src/cuirass/database.scm b/src/cuirass/database.scm
> index b4b1652..7788ac9 100644
> --- a/src/cuirass/database.scm
> +++ b/src/cuirass/database.scm
[…]
> (define (db-add-evaluation db eval)
> (sqlite-exec db "\
> INSERT INTO Evaluations (specification, commits) VALUES ("
> @@ -384,27 +356,39 @@ string."
> (define (db-add-build db build)
> "Store BUILD in database DB. BUILD eventual outputs are stored
> in the OUTPUTS table."
> - (let* ((build-exec
> - (sqlite-exec db "\
> -INSERT INTO Builds (derivation, evaluation, log, status, timestamp, starttime, stoptime)\
> - VALUES ("
> - (assq-ref build #:derivation) ", "
> - (assq-ref build #:eval-id) ", "
> - (assq-ref build #:log) ", "
> - (or (assq-ref build #:status)
> - (build-status scheduled)) ", "
> - (or (assq-ref build #:timestamp) 0) ", "
> - (or (assq-ref build #:starttime) 0) ", "
> - (or (assq-ref build #:stoptime) 0) ");"))
> - (build-id (last-insert-rowid db)))
> - (for-each (lambda (output)
> - (match output
> - ((name . path)
> - (sqlite-exec db "\
> -INSERT INTO Outputs (build, name, path) VALUES ("
> - build-id ", " name ", " path ");"))))
> - (assq-ref build #:outputs))
> - build-id))
> + (catch 'sqlite-error
> + (lambda ()
> + (sqlite-exec db "
> +INSERT INTO Builds (derivation, evaluation, job_name, system, nix_name, log,
> +status, timestamp, starttime, stoptime)
> +VALUES ("
> + (assq-ref build #:derivation) ", "
> + (assq-ref build #:eval-id) ", "
> + (assq-ref build #:job-name) ", "
> + (assq-ref build #:system) ", "
> + (assq-ref build #:nix-name) ", "
> + (assq-ref build #:log) ", "
> + (or (assq-ref build #:status)
> + (build-status scheduled)) ", "
> + (or (assq-ref build #:timestamp) 0) ", "
> + (or (assq-ref build #:starttime) 0) ", "
> + (or (assq-ref build #:stoptime) 0) ");")
> + (let ((derivation (assq-ref build #:derivation)))
> + (for-each (lambda (output)
> + (match output
> + ((name . path)
> + (sqlite-exec db "\
> +INSERT INTO Outputs (derivation, name, path) VALUES ("
> + derivation ", " name ", " path ");"))))
> + (assq-ref build #:outputs))
> + derivation))
This procedure is called when a build is scheduled, isn’t it? The
docstring says “BUILD eventual outputs are stored in the OUTPUTS table.”
– does this mean the names of the *expected* output directories are
recorded in Outputs? They don’t exist at this point, right?
> + (lambda (key who code message . rest)
> + ;; If we get a unique-constraint-failed error, that means we have
> + ;; already inserted the same build. That happens when several jobs
> + ;; produce the same derivation, and we can ignore it.
> + (if (= code SQLITE_CONSTRAINT_PRIMARYKEY)
> + #f
> + (apply throw key who code rest)))))
Okay.
Can we prevent this from happening in the first place? I feel a little
uncomfortable about performing an operation that we expect to cause
primary key errors regularly.
> diff --git a/src/schema.sql b/src/schema.sql
> index eb0f7e9..0452495 100644
> --- a/src/schema.sql
> +++ b/src/schema.sql
[…]
> --- Builds are not in a one to one relationship with derivations in order to
> --- keep track of non deterministic compilations.
Is this comment still correct considering that the derivation is now the
primary key of the Builds table?
> CREATE TABLE Builds (
> - id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
> - derivation TEXT NOT NULL,
> + derivation TEXT NOT NULL PRIMARY KEY,
> evaluation INTEGER NOT NULL,
> + job_name TEXT NOT NULL,
> + system TEXT NOT NULL,
> + nix_name TEXT NOT NULL,
> log TEXT NOT NULL,
> status INTEGER NOT NULL,
> timestamp INTEGER NOT NULL,
> starttime INTEGER NOT NULL,
> stoptime INTEGER NOT NULL,
> - FOREIGN KEY (derivation) REFERENCES Derivations (derivation),
> FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
> );
Thanks again!
--
Ricardo
next prev parent reply other threads:[~2018-08-17 10:54 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAMSS15C0nNEPqQKjRt9=-JKFvrGZsKBtMOxju65p_y88EzOZgg@mail.gmail.com>
[not found] ` <87vac3twbe.fsf@gnu.org>
[not found] ` <87o9hog2ye.fsf@elephly.net>
[not found] ` <CAMSS15CQwypdFafPG1ii-CgMpxYxLtcbx8hHTvDvyBS+6xNSxA@mail.gmail.com>
[not found] ` <87d0xyn9zs.fsf@elephly.net>
[not found] ` <CAMSS15CGdzb8=-Oz6z6bmeWjrnT85PL+q7DLUFw-9E4_d4Y6pw@mail.gmail.com>
[not found] ` <87d0xswvls.fsf@elephly.net>
[not found] ` <CAMSS15CDGat-pFjiz2vrkvb14qWnY4rbCW-d8KSzY6MO7WzT_g@mail.gmail.com>
[not found] ` <87r2m4ntk4.fsf@mdc-berlin.de>
[not found] ` <CAMSS15D42gCV-UneBBJWngdfi-jG4JfWWh6NWhWMiet0Y=bUsg@mail.gmail.com>
[not found] ` <87tvqxy4i9.fsf@elephly.net>
[not found] ` <CAMSS15DThnLO+YEVaBmJ9ozMeu4mO1rHAdXHgZ8K+Csu40pORQ@mail.gmail.com>
[not found] ` <87in78hxo2.fsf@elephly.net>
2018-07-17 19:31 ` GSoC: Adding a web interface similar to the Hydra web interface Clément Lassieur
2018-07-17 22:32 ` bug#32190: Cuirass doesn't check if two subsequent jobs yield the same derivation Clément Lassieur
2018-07-24 10:05 ` Ludovic Courtès
2018-08-04 16:03 ` bug#32190: [PATCH] database: Merge Derivations into Builds table Clément Lassieur
2018-08-04 16:09 ` Clément Lassieur
2018-08-08 12:13 ` Clément Lassieur
2018-08-14 16:57 ` Clément Lassieur
2018-08-14 19:04 ` Ricardo Wurmus [this message]
2018-08-15 18:57 ` Clément Lassieur
2018-08-16 21:00 ` Clément Lassieur
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87h8jwvkg8.fsf@elephly.net \
--to=rekado@elephly.net \
--cc=32190@debbugs.gnu.org \
--cc=clement@lassieur.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).