From: "Ludovic Courtès" <ludo@gnu.org>
To: Caleb Ristvedt <caleb.ristvedt@cune.org>
Cc: Danny Milosavljevic <dannym@scratchpost.org>, 41658@debbugs.gnu.org
Subject: [bug#41658] [PATCH] fixes / improvements for (guix store database)
Date: Tue, 09 Jun 2020 10:42:00 +0200 [thread overview]
Message-ID: <87h7vkmy07.fsf@gnu.org> (raw)
In-Reply-To: <87o8pu5cjx.fsf@cune.org> (Caleb Ristvedt's message of "Mon, 08 Jun 2020 00:52:50 -0500")
Hi,
Caleb Ristvedt <caleb.ristvedt@cune.org> skribis:
[...]
>> Nice. It would be great if you could report it upstream (Danny and/or
>> myself can then patch it directly in guile-sqlite3 and push out a
>> release) and refer to the issue from here.
>
> Reported as https://notabug.org/guile-sqlite3/guile-sqlite3/issues/12,
> with corresponding pull request
> https://notabug.org/guile-sqlite3/guile-sqlite3/pulls/13.
Awesome, thank you.
>> We can have this patch locally in the meantime, unless it would break
>> once the new guile-sqlite3 is out. WDYT?
>
> I've attached an updated patch series that both includes the
> guile-sqlite3 fix as a patch to the guile-sqlite3 package and adopts the
> workaround for situations where older guile-sqlite3's must be used (for
> example, when building guix from scratch on foreign distros that haven't
> incorporated the fix yet). The only changes are the addition of the
> now-second patch and fixing up some spacing in the comment in the first
> patch.
OK.
>>> * guix/store/database.scm (call-with-savepoint): new procedure.
>>> (update-or-insert): use call-with-savepoint to ensure the read and the
>>> insert/update occur within the same transaction.
>>
>> That’s a bit beyond my understanding, but I think you can also push this
>> one. :-)
>
> Basically, it's like combining the body of two separate compare-and-swap
> loops into a single compare-and-swap loop. This ensures that the view is
> consistent (since if it isn't, the "compare" will fail and we'll
> retry). It addresses a problem that doesn't exist in practice yet, since
> update-or-insert is only called from within a call-with-transaction
> currently. But if someone ever wanted to call it from outside of a
> call-with-transaction, this would ensure that it still worked correctly.
Makes sense, thanks for explaining.
> Related question: does berlin export /var/guix over NFS as per
> http://hpc.guix.info/blog/2017/11/installing-guix-on-a-cluster? If so,
> that could interact poorly with our use of WAL mode:
No, it doesn’t. (Also, in the setup described above, there’s only one
guix-daemon instance and it accesses the database via the local file
system.)
> From 614213c80a7ea15f7aab9502e6c33206ac089d05 Mon Sep 17 00:00:00 2001
> From: Caleb Ristvedt <caleb.ristvedt@cune.org>
> Date: Mon, 1 Jun 2020 18:50:07 -0500
> Subject: [PATCH 1/5] database: work around guile-sqlite3 bug preventing
> statement reset
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> guile-sqlite3 provides statement caching, making it unnecessary for sqlite to
> keep re-preparing statements that are frequently used. Unfortunately it
> doesn't quite emulate the semantics of sqlite_finalize properly, because it
> doesn't cause a commit if the statement being finalized is the last "active"
> statement (see https://notabug.org/guile-sqlite3/guile-sqlite3/issues/12). We
> work around this by wrapping sqlite-finalize with our own version that ensures
> sqlite-reset is called, which does The Right Thing™.
>
> * guix/store/database.scm (sqlite-finalize): new procedure that shadows the
> sqlite-finalize from (sqlite3).
[...]
> +(define (sqlite-finalize stmt)
> + ;; As of guile-sqlite3 0.1.0, cached statements aren't reset when
> + ;; sqlite-finalize is invoked on them (see
> + ;; https://notabug.org/guile-sqlite3/guile-sqlite3/issues/12). This can
> + ;; cause problems with automatically-started transactions:
I think it’s enough to link to the upstream issue, which has the problem
well documented.
> From e3cf7be4491f465d3041933596d3caad1ea64e83 Mon Sep 17 00:00:00 2001
> From: Caleb Ristvedt <caleb.ristvedt@cune.org>
> Date: Sun, 7 Jun 2020 22:30:41 -0500
> Subject: [PATCH 2/5] gnu: guile-sqlite3: add patch to fix sqlite-finalize bug
>
> Adds patch that fixes
> https://notabug.org/guile-sqlite3/guile-sqlite3/issues/12. This can be
> discarded once the patch is integrated into the next guile-sqlite3 release.
> Note that the patch is identical to the pull request at
> https://notabug.org/guile-sqlite3/guile-sqlite3/pulls/13.
>
> * gnu/packages/patches/guile-sqlite3-reset-on-sqlite-finalize.patch: new
> patch.
> * gnu/packages/guile.scm (guile-sqlite3): use it.
> * gnu/local.mk (dist_patch_DATA): add it.
I’d skip it: we have a workaround and the release may be out soon.
Danny, thoughts on getting a new release out?
The rest is still fine with me, thank you!
Ludo’.
prev parent reply other threads:[~2020-06-09 8:44 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-02 6:31 [bug#41658] [PATCH] fixes / improvements for (guix store database) Caleb Ristvedt
2020-06-04 16:40 ` Ludovic Courtès
2020-06-04 17:00 ` Danny Milosavljevic
2020-06-05 16:19 ` Ludovic Courtès
2020-06-08 5:52 ` Caleb Ristvedt
2020-06-09 8:42 ` Ludovic Courtès [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87h7vkmy07.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=41658@debbugs.gnu.org \
--cc=caleb.ristvedt@cune.org \
--cc=dannym@scratchpost.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).