From: Austin Clements <amdragon@MIT.EDU>
To: David Bremner <bremner@debian.org>
Cc: Olly Betts <olly@survex.com>, Notmuch Mail <notmuch@notmuchmail.org>
Subject: Re: More ideas about logging.
Date: Thu, 15 Dec 2011 23:07:22 -0500 [thread overview]
Message-ID: <20111216040722.GC12245@mit.edu> (raw)
In-Reply-To: <87obv9i7y3.fsf@zancas.localnet>
Quoth David Bremner on Dec 15 at 10:09 pm:
> Assume we have routines read_metadata and write_metadata that read and
> write to the xapian database metadata (in real life, I think we might
> need to decide in advance exactly what will be written there).
>
> when we create a database
>
> write_metadata('log_write',0)
> write_metadata('log_read',0) // more about this later
>
> To carry out database operation X with logging, we do the following
>
> begin_atomic
>
> txn=read_metadata('last_written')
>
> X
>
> // begin dangerzone
> fprintf(logfile,"%d %s",num+1,stuff) // or whatever.
>
> write_metadata('last_written', num+1)
>
> end_atomic
> //end dangerzone
The trouble with this approach is that the OS doesn't have to flush
logfile to the disk platters in any particular order relative to the
updates to Xapian. So, after someone trips over your plug, you could
come back with Xapian saying you have 500 log entries when your
logfile comes back with only 20. The only way I know of to fix this
is to fsync after the logfile write, which would obviously have
performance issues. But maybe there are cleverer ways?
next prev parent reply other threads:[~2011-12-16 4:06 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-16 2:09 More ideas about logging David Bremner
2011-12-16 4:07 ` Austin Clements [this message]
2011-12-16 11:56 ` David Bremner
2011-12-18 18:34 ` David Bremner
2011-12-18 20:22 ` Tom Prince
2011-12-20 20:25 ` David Bremner
2012-10-12 16:28 ` Ethan Glasser-Camp
2011-12-16 7:16 ` Michael Hudson-Doyle
2011-12-16 12:02 ` David Bremner
2011-12-18 21:53 ` Michael Hudson-Doyle
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111216040722.GC12245@mit.edu \
--to=amdragon@mit.edu \
--cc=bremner@debian.org \
--cc=notmuch@notmuchmail.org \
--cc=olly@survex.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).