Cuirass and duplicate derivations

Cuirass and duplicate derivations

[Ludovic Courtès]

Hello!

Cuirass is almost up and running on the new machine!  :-)

The last problem I experienced is this:

--8<---------------cut here---------------start------------->8---
evaluate 'gfortran-4.9.4.x86_64-linux': 0.136 seconds
evaluate 'gfortran-4.9.4.x86_64-linux': 0.000 seconds
Backtrace:
In ice-9/boot-9.scm:
 160: 12 [catch #t #<catch-closure 1b9bc40> ...]
In unknown file:
   ?: 11 [apply-smob/1 #<catch-closure 1b9bc40>]
In ice-9/boot-9.scm:
  66: 10 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 9 [eval # #]
In ice-9/r4rs.scm:
  90: 8 [dynamic-wind #<procedure a26a600 at ice-9/boot-9.scm:1136:2 _> ...]
In ice-9/eval.scm:
 481: 7 [lp (#<fluid 22> #<fluid 34>) ("/var/run/cuirass/cuirass.db" #f)]
 481: 6 [lp (#<fluid 34>) (#f)]
 432: 5 [eval # #]
 387: 4 [eval # #]
In ice-9/boot-9.scm:
 705: 3 [map #<procedure c3bbdb0 at ice-9/eval.scm:416:20 (a)> #]
In ice-9/eval.scm:
 432: 2 [eval # #]
In src/cuirass/database.scm:
  54: 1 [sqlite-exec # ...]
In ice-9/eval.scm:
 432: 0 [eval # #]

ice-9/eval.scm:432:17: In procedure eval:
ice-9/eval.scm:432:17: Throw to key `sqlite-error' with args `(#f 1555 "UNIQUE constraint failed: Derivations.derivation, Derivations.evaluation")'.
--8<---------------cut here---------------end--------------->8---

… which commit 7355634db3ccf0d86f8e34c4aea37392c1a0ab0a fixes.

Then there was another one:

--8<---------------cut here---------------start------------->8---
evaluate 'wine-1.9.24.i686-linux': 0.205 seconds
Backtrace:
In ice-9/boot-9.scm:
 160: 12 [catch #t #<catch-closure 9efb20> ...]
In unknown file:
   ?: 11 [apply-smob/1 #<catch-closure 9efb20>]
In ice-9/boot-9.scm:
  66: 10 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 9 [eval # #]
In ice-9/r4rs.scm:
  90: 8 [dynamic-wind #<procedure 9104c20 at ice-9/boot-9.scm:1136:2 _> ...]
In ice-9/eval.scm:
 481: 7 [lp (#<fluid 22> #<fluid 34>) ("/var/run/cuirass/cuirass.db" #f)]
 481: 6 [lp (#<fluid 34>) (#f)]
 432: 5 [eval # #]
 387: 4 [eval # #]
In ice-9/boot-9.scm:
 703: 3 [map #<procedure b0913f0 at ice-9/eval.scm:416:20 (a)> #]
In ice-9/eval.scm:
 432: 2 [eval # #]
In src/cuirass/database.scm:
  54: 1 [sqlite-exec # ...]
In ice-9/eval.scm:
 432: 0 [eval # #]

ice-9/eval.scm:432:17: In procedure eval:
ice-9/eval.scm:432:17: Throw to key `sqlite-error' with args `(#f 1555 "UNIQUE constraint failed: Derivations.derivation, Derivations.evaluation")'.
--8<---------------cut here---------------end--------------->8---

This is because Wine is always built for i686-linux, even on x86_64,
hence the same .drv.  So I hacked my way to ignore Wine.

But then ‘cargo-bootstrap’ showed the same problem.

Hence this message.  :-)

What change would you suggest to solve this problem?  It would be best
if Cuirass allowed several jobs building the same derivations (the key
could be an autoincrement counter instead of the (drv,eval) pair maybe?)

TIA!

Ludo’.

Re: Cuirass and duplicate derivations

[Mathieu Lirzin]

Hello,

ludo@gnu.org (Ludovic Courtès) writes:

> Cuirass is almost up and running on the new machine!  :-)

Great work!

> The last problem I experienced is this:
>
> evaluate 'gfortran-4.9.4.x86_64-linux': 0.136 seconds
> evaluate 'gfortran-4.9.4.x86_64-linux': 0.000 seconds
> Backtrace:
> In ice-9/boot-9.scm:
>  160: 12 [catch #t #<catch-closure 1b9bc40> ...]
> In unknown file:
>    ?: 11 [apply-smob/1 #<catch-closure 1b9bc40>]
> In ice-9/boot-9.scm:
>   66: 10 [call-with-prompt prompt0 ...]
> In ice-9/eval.scm:
>  432: 9 [eval # #]
> In ice-9/r4rs.scm:
>   90: 8 [dynamic-wind #<procedure a26a600 at ice-9/boot-9.scm:1136:2 _> ...]
> In ice-9/eval.scm:
>  481: 7 [lp (#<fluid 22> #<fluid 34>) ("/var/run/cuirass/cuirass.db" #f)]
>  481: 6 [lp (#<fluid 34>) (#f)]
>  432: 5 [eval # #]
>  387: 4 [eval # #]
> In ice-9/boot-9.scm:
>  705: 3 [map #<procedure c3bbdb0 at ice-9/eval.scm:416:20 (a)> #]
> In ice-9/eval.scm:
>  432: 2 [eval # #]
> In src/cuirass/database.scm:
>   54: 1 [sqlite-exec # ...]
> In ice-9/eval.scm:
>  432: 0 [eval # #]
>
> ice-9/eval.scm:432:17: In procedure eval:
> ice-9/eval.scm:432:17: Throw to key `sqlite-error' with args `(#f 1555 "UNIQUE constraint failed: Derivations.derivation, Derivations.evaluation")'.
>
> … which commit 7355634db3ccf0d86f8e34c4aea37392c1a0ab0a fixes.
>
> Then there was another one:
>
> evaluate 'wine-1.9.24.i686-linux': 0.205 seconds
> Backtrace:
> In ice-9/boot-9.scm:
>  160: 12 [catch #t #<catch-closure 9efb20> ...]
> In unknown file:
>    ?: 11 [apply-smob/1 #<catch-closure 9efb20>]
> In ice-9/boot-9.scm:
>   66: 10 [call-with-prompt prompt0 ...]
> In ice-9/eval.scm:
>  432: 9 [eval # #]
> In ice-9/r4rs.scm:
>   90: 8 [dynamic-wind #<procedure 9104c20 at ice-9/boot-9.scm:1136:2 _> ...]
> In ice-9/eval.scm:
>  481: 7 [lp (#<fluid 22> #<fluid 34>) ("/var/run/cuirass/cuirass.db" #f)]
>  481: 6 [lp (#<fluid 34>) (#f)]
>  432: 5 [eval # #]
>  387: 4 [eval # #]
> In ice-9/boot-9.scm:
>  703: 3 [map #<procedure b0913f0 at ice-9/eval.scm:416:20 (a)> #]
> In ice-9/eval.scm:
>  432: 2 [eval # #]
> In src/cuirass/database.scm:
>   54: 1 [sqlite-exec # ...]
> In ice-9/eval.scm:
>  432: 0 [eval # #]
>
> ice-9/eval.scm:432:17: In procedure eval:
> ice-9/eval.scm:432:17: Throw to key `sqlite-error' with args `(#f 1555 "UNIQUE constraint failed: Derivations.derivation, Derivations.evaluation")'.
>
> This is because Wine is always built for i686-linux, even on x86_64,
> hence the same .drv.  So I hacked my way to ignore Wine.
>
> But then ‘cargo-bootstrap’ showed the same problem.
>
> Hence this message.  :-)
>
> What change would you suggest to solve this problem?  It would be best
> if Cuirass allowed several jobs building the same derivations (the key
> could be an autoincrement counter instead of the (drv,eval) pair maybe?)

Seems reasonable to me.  I will do that.

Thanks.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Re: Cuirass and duplicate derivations

[Mathieu Lirzin]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

Mathieu Lirzin <mthl@gnu.org> writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> What change would you suggest to solve this problem?  It would be best
>> if Cuirass allowed several jobs building the same derivations (the key
>> could be an autoincrement counter instead of the (drv,eval) pair maybe?)
>
> Seems reasonable to me.  I will do that.
> Thanks.

After a second thought, I think maybe it is sufficient to just ignore
when a derivation is added twice by an evaluation.  As a consequence
'cuirass' will only try to realize the derivation once per evaluation.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-database-db-add-derivation-Don-t-try-to-add-a-deriva.patch --]
[-- Type: text/x-diff, Size: 1036 bytes --]

From 568d0e1b0866a45e95440d17b6e8f1740cc23e3f Mon Sep 17 00:00:00 2001
From: Mathieu Lirzin <mthl@gnu.org>
Date: Mon, 9 Jan 2017 01:29:48 +0100
Subject: [PATCH] database: db-add-derivation: Don't try to add a derivation
 twice.

* src/cuirass/database.scm (db-add-derivation): Ignore if JOB is already
present in DB.
---
 src/cuirass/database.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/cuirass/database.scm b/src/cuirass/database.scm
index 870cdc0..702e643 100644
--- a/src/cuirass/database.scm
+++ b/src/cuirass/database.scm
@@ -147,7 +147,7 @@ INSERT OR IGNORE INTO Specifications (repo_name, url, load_path, file, \
 (define (db-add-derivation db job)
   "Store a derivation result in database DB and return its ID."
   (sqlite-exec db "\
-INSERT INTO Derivations (derivation, job_name, evaluation)\
+INSERT OR IGNORE INTO Derivations (derivation, job_name, evaluation)\
   VALUES ('~A', '~A', '~A');"
                (assq-ref job #:derivation)
                (assq-ref job #:job-name)
-- 
2.11.0


[-- Attachment #3: Type: text/plain, Size: 92 bytes --]


WDYT?

Thanks.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Re: Cuirass and duplicate derivations

[Ludovic Courtès]

Hello,

Mathieu Lirzin <mthl@gnu.org> skribis:

> Mathieu Lirzin <mthl@gnu.org> writes:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> What change would you suggest to solve this problem?  It would be best
>>> if Cuirass allowed several jobs building the same derivations (the key
>>> could be an autoincrement counter instead of the (drv,eval) pair maybe?)
>>
>> Seems reasonable to me.  I will do that.
>> Thanks.
>
> After a second thought, I think maybe it is sufficient to just ignore
> when a derivation is added twice by an evaluation.  As a consequence
> 'cuirass' will only try to realize the derivation once per evaluation.

Even better, yes.

> From 568d0e1b0866a45e95440d17b6e8f1740cc23e3f Mon Sep 17 00:00:00 2001
> From: Mathieu Lirzin <mthl@gnu.org>
> Date: Mon, 9 Jan 2017 01:29:48 +0100
> Subject: [PATCH] database: db-add-derivation: Don't try to add a derivation
>  twice.
>
> * src/cuirass/database.scm (db-add-derivation): Ignore if JOB is already
> present in DB.

OK!

We can update the ‘cuirass’ package in Guix afterwards.

Thank you!

Ludo’.