From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Cl=C3=A9ment?= Lassieur Subject: bug#32190: Cuirass doesn't check if two subsequent jobs yield the same derivation Date: Wed, 18 Jul 2018 00:32:02 +0200 Message-ID: <87efg1ijdp.fsf@lassieur.org> References: <87k1ptirr0.fsf@lassieur.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:46451) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffYWn-0008QC-7k for bug-guix@gnu.org; Tue, 17 Jul 2018 18:33:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffYWk-0000lV-3I for bug-guix@gnu.org; Tue, 17 Jul 2018 18:33:05 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:40904) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ffYWj-0000lF-UM for bug-guix@gnu.org; Tue, 17 Jul 2018 18:33:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ffYWj-0004uW-LR for bug-guix@gnu.org; Tue, 17 Jul 2018 18:33:01 -0400 In-Reply-To: <87k1ptirr0.fsf@lassieur.org> Sender: "Debbugs-submit" Resent-Message-ID: List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: 32190@debbugs.gnu.org Consider the following table: --8<---------------cut here---------------start------------->8--- CREATE TABLE Derivations ( derivation TEXT NOT NULL, evaluation INTEGER NOT NULL, job_name TEXT NOT NULL, system TEXT NOT NULL, nix_name TEXT NOT NULL, PRIMARY KEY (derivation, evaluation), FOREIGN KEY (evaluation) REFERENCES Evaluations (id) ); --8<---------------cut here---------------end--------------->8--- And the following code: --8<---------------cut here---------------start------------->8--- (define (db-add-derivation db job) "Store a derivation result in database DB and return its ID." (catch 'sqlite-error (lambda () (sqlite-exec db "\ INSERT INTO Derivations (derivation, job_name, system, nix_name, evaluation= )\ VALUES (" (assq-ref job #:derivation) ", " (assq-ref job #:job-name) ", " (assq-ref job #:system) ", " (assq-ref job #:nix-name) ", " (assq-ref job #:eval-id) ");") (last-insert-rowid db)) (lambda (key who code message . rest) ;; If we get a unique-constraint-failed error, that means we have ;; already inserted the same (derivation,eval-id) tuple. That happens ;; when several jobs produce the same derivation, and we can ignore i= t. (if (=3D code SQLITE_CONSTRAINT_PRIMARYKEY) (sqlite-exec db "SELECT * FROM Derivations WHERE derivation=3D" (assq-ref job #:derivation) ";") (apply throw key who code rest))))) --8<---------------cut here---------------end--------------->8--- I think the above constraint can't happen because by definition a new job (for the same job_name) is produced at each evaluation. So eval-id will be incremented every time. Also, the docs (and a comment in schema.sql) says: Builds are not in a one to one relationship with derivations in order to keep track of non deterministic compilations. But I think it doesn't make sense, because Guix won't try to build twice the same thing unless '--check' is used (which obviously isn't the case). So not only we have a huge Derivations table full of identical items, but we also ask Guix to build them and we store the results in the Builds table... Maybe the solution is to replace the (derivation, evaluation) primary key with (derivation), and only build the newly added derivations. WDYT? Cl=C3=A9ment