unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#42023] [PATCH] database: register-items: reduce transaction scope.
@ 2020-06-23 16:36 Christopher Baines
  2020-06-23 22:15 ` Ludovic Courtès
  0 siblings, 1 reply; 9+ messages in thread
From: Christopher Baines @ 2020-06-23 16:36 UTC (permalink / raw)
  To: 42023

It was made transactional in a4678c6ba18d8dbd79d931f80426eebf61be7ebe, with
the reasoning to prevent broken intermediate states from being visible. I
think this means something like an entry being in ValidPaths, but the Refs not
being inserted.

Using a transaction for this makes sense, but I think using one single
transaction for the whole register-items call is unnecessary to avoid broken
states from being visible, and could block other writes to the store database
while register-items is running. Because the deduplication and resetting
timestamps happens within the transaction as well, even though these things
don't involve the database, writes to the database will still be blocked while
this is happening.

To reduce the potential for register-items to block other writers to the
database for extended periods, this commit moves the transaction to just wrap
the call to sqlite-register. This is the one place where writes occur, so that
should prevent the broken intermediate states issue above. The one difference
this will make is some of the registered items will be visible to other
connections while others may be still being added. I think this is OK, as it's
equivalent to just registering different items.

* guix/store/database.scm (register-items): Reduce transaction scope.
---
 guix/store/database.scm | 37 +++++++++++++++++++------------------
 1 file changed, 19 insertions(+), 18 deletions(-)

diff --git a/guix/store/database.scm b/guix/store/database.scm
index a38e4d7e52..767335bc0f 100644
--- a/guix/store/database.scm
+++ b/guix/store/database.scm
@@ -441,24 +441,25 @@ in the database; #f means \"now\".  Write a progress report to LOG-PORT."
       (when reset-timestamps?
         (reset-timestamps real-file-name))
       (let-values (((hash nar-size) (nar-sha256 real-file-name)))
-        (sqlite-register db #:path to-register
-                         #:references (store-info-references item)
-                         #:deriver (store-info-deriver item)
-                         #:hash (string-append "sha256:"
-                                               (bytevector->base16-string hash))
-                         #:nar-size nar-size
-                         #:time registration-time)
+        (call-with-retrying-transaction db
+            (lambda ()
+              (sqlite-register db #:path to-register
+                               #:references (store-info-references item)
+                               #:deriver (store-info-deriver item)
+                               #:hash (string-append
+                                       "sha256:"
+                                       (bytevector->base16-string hash))
+                               #:nar-size nar-size
+                               #:time registration-time)))
         (when deduplicate?
           (deduplicate real-file-name hash #:store store-dir)))))
 
-  (call-with-retrying-transaction db
-      (lambda ()
-        (let* ((prefix   (format #f "registering ~a items" (length items)))
-               (progress (progress-reporter/bar (length items)
-                                                prefix log-port)))
-          (call-with-progress-reporter progress
-            (lambda (report)
-              (for-each (lambda (item)
-                          (register db item)
-                          (report))
-                        items)))))))
+  (let* ((prefix   (format #f "registering ~a items" (length items)))
+         (progress (progress-reporter/bar (length items)
+                                          prefix log-port)))
+    (call-with-progress-reporter progress
+      (lambda (report)
+        (for-each (lambda (item)
+                    (register db item)
+                    (report))
+                  items)))))
-- 
2.26.2





^ permalink raw reply related	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2020-09-16 20:38 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-23 16:36 [bug#42023] [PATCH] database: register-items: reduce transaction scope Christopher Baines
2020-06-23 22:15 ` Ludovic Courtès
2020-06-24 17:51   ` Caleb Ristvedt
2020-06-25  7:45     ` Ludovic Courtès
2020-07-26 15:31       ` Ludovic Courtès
2020-08-09  4:13         ` Caleb Ristvedt
2020-09-14  8:58           ` Ludovic Courtès
2020-09-15 20:29             ` [bug#42023] [PATCH] Retry deduplication on ENOENT Caleb Ristvedt
2020-09-16 20:37               ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).