unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [bug] notmuch doesn't commit changes before an open transaction on close
@ 2015-10-25 21:02 Steven Allen
  2017-03-12  0:33 ` David Bremner
  2021-12-26  0:13 ` David Bremner
  0 siblings, 2 replies; 3+ messages in thread
From: Steven Allen @ 2015-10-25 21:02 UTC (permalink / raw)
  To: notmuch

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

Notmuch claims to commit changes made before an open transaction on
close but actually throws them away (according to the documentation).

According to the notmuch documentation,

> For writable databases, notmuch_database_close commits all changes
> to disk before closing the database.  If the caller is currently in
> an atomic section (there was a notmuch_database_begin_atomic
> without a matching notmuch_database_end_atomic), this will discard
> changes made in that atomic section (but still commit changes made
> prior to entering the atomic section).

However, this isn't true. Notmuch atomic transactions don't flush on
entry so, this comment from the xapian documentation applies:

> If you're applying atomic groups of changes and only wish to ensure
> that each group is either applied or not applied, then you can prevent
> the automatic commit() before and after the transaction by starting
> the transaction with begin_transaction(false). However, if
> cancel_transaction is called (or if commit_transaction isn't called
> before the WritableDatabase object is destroyed) then any changes
> which were pending before the transaction began will also be
> discarded.

source: http://xapian.org/docs/apidoc/html/classXapian_1_1WritableDatabase.html

This means that, in theory at least, xapian could throw away *all*
changes to the database if a transaction is open.

-- 
Steven Allen
(310) 433-5865
((Do Not Email <honeypot@stebalien.com>))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [bug] notmuch doesn't commit changes before an open transaction on close
  2015-10-25 21:02 [bug] notmuch doesn't commit changes before an open transaction on close Steven Allen
@ 2017-03-12  0:33 ` David Bremner
  2021-12-26  0:13 ` David Bremner
  1 sibling, 0 replies; 3+ messages in thread
From: David Bremner @ 2017-03-12  0:33 UTC (permalink / raw)
  To: Steven Allen, notmuch

Steven Allen <steven@stebalien.com> writes:

> Notmuch claims to commit changes made before an open transaction on
> close but actually throws them away (according to the documentation).
>
> According to the notmuch documentation,
>
>> For writable databases, notmuch_database_close commits all changes
>> to disk before closing the database.  If the caller is currently in
>> an atomic section (there was a notmuch_database_begin_atomic
>> without a matching notmuch_database_end_atomic), this will discard
>> changes made in that atomic section (but still commit changes made
>> prior to entering the atomic section).
>
> However, this isn't true. Notmuch atomic transactions don't flush on
> entry so, this comment from the xapian documentation applies:
>
>> If you're applying atomic groups of changes and only wish to ensure
>> that each group is either applied or not applied, then you can prevent
>> the automatic commit() before and after the transaction by starting
>> the transaction with begin_transaction(false). However, if
>> cancel_transaction is called (or if commit_transaction isn't called
>> before the WritableDatabase object is destroyed) then any changes
>> which were pending before the transaction began will also be
>> discarded.
>
> source: http://xapian.org/docs/apidoc/html/classXapian_1_1WritableDatabase.html
>
> This means that, in theory at least, xapian could throw away *all*
> changes to the database if a transaction is open.
>

I was curioius what would happen if we did commit on entry, but the
performance loss is pretty catastrophic. The small performance test
corpus runs 20 times slower, and does about 200 times as many writes, so
it would probably be even worse on a machine without SSD.  Perhaps we
should just provide notmuch_database_commit and let people call that,
although it might be just as easy to wrap whatever changes are being
committed in a begin/end atomic.


[1]:

diff --git a/lib/database.cc b/lib/database.cc
index a679cbab..da67a5df 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1720,7 +1720,7 @@ notmuch_database_begin_atomic (notmuch_database_t *notmuch)
        return NOTMUCH_STATUS_UPGRADE_REQUIRED;
 
     try {
-       (static_cast <Xapian::WritableDatabase *> (notmuch->xapian_db))->begin_transaction (false);
+       (static_cast <Xapian::WritableDatabase *> (notmuch->xapian_db))->begin_transaction (true);
     } catch (const Xapian::Error &error) {
        _notmuch_database_log (notmuch, "A Xapian exception occurred beginning transaction: %s.\n",
                 error.get_msg().c_str());

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [bug] notmuch doesn't commit changes before an open transaction on close
  2015-10-25 21:02 [bug] notmuch doesn't commit changes before an open transaction on close Steven Allen
  2017-03-12  0:33 ` David Bremner
@ 2021-12-26  0:13 ` David Bremner
  1 sibling, 0 replies; 3+ messages in thread
From: David Bremner @ 2021-12-26  0:13 UTC (permalink / raw)
  To: Steven Allen, notmuch

Steven Allen <steven@stebalien.com> writes:

> Notmuch claims to commit changes made before an open transaction on
> close but actually throws them away (according to the documentation).
>
> According to the notmuch documentation,
>
>> For writable databases, notmuch_database_close commits all changes
>> to disk before closing the database.  If the caller is currently in
>> an atomic section (there was a notmuch_database_begin_atomic
>> without a matching notmuch_database_end_atomic), this will discard
>> changes made in that atomic section (but still commit changes made
>> prior to entering the atomic section).
>
> However, this isn't true. Notmuch atomic transactions don't flush on
> entry so, this comment from the xapian documentation applies:
>
>> If you're applying atomic groups of changes and only wish to ensure
>> that each group is either applied or not applied, then you can prevent
>> the automatic commit() before and after the transaction by starting
>> the transaction with begin_transaction(false). However, if
>> cancel_transaction is called (or if commit_transaction isn't called
>> before the WritableDatabase object is destroyed) then any changes
>> which were pending before the transaction began will also be
>> discarded.
>
> source: http://xapian.org/docs/apidoc/html/classXapian_1_1WritableDatabase.html
>
> This means that, in theory at least, xapian could throw away *all*
> changes to the database if a transaction is open.

The docs are corrected, and notmuch now does periodic automatic commits
to reduce the amount of lost work when interrupted. At the moment I
think that is the best we can do, so I'm marking this as fixed.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-12-26  0:13 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-10-25 21:02 [bug] notmuch doesn't commit changes before an open transaction on close Steven Allen
2017-03-12  0:33 ` David Bremner
2021-12-26  0:13 ` David Bremner

Code repositories for project(s) associated with this inbox:

	notmuch.git.git (no URL configured)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).