please eat my data!

please eat my data!

[Sebastian Spaeth]

fsync is really killing xapian (and notmuch). What suffers, are the
boolean prefixes (tag, id, and thread). Using libeatmydata (which
disables fsync) shows a 10x speedup for tagging. The speedup is only
factor 2 for e.g. from: searches. This is ext4 on recent stock
Ubuntu. Given that search by tag and thread are performed really often
(each time I advance a thread, for example), that really hurts.

With a warm file cache and a thread containing 11 messages:

---------------------------------------------------
time notmuch tag +test -- thread:0000000000000f4e
real	0m0.677s
user	0m0.030s
sys	0m0.020s
---------------------------------------------------
time LD_PRELOAD=./libeatmydata.so notmuch tag +test -- thread:0000000000000f4e

real	0m0.040s
user	0m0.020s
sys	0m0.020s
---------------------------------------------------

However tagging ~850 messages based on a from search is "ONLY" factor 2:
------------------------------------------------------
time notmuch tag +test -- from:"sebastian@sspaeth.de"

real	0m2.355s
user	0m1.240s
sys	0m0.040s
---------------------------------------------------
time LD_PRELOAD=./libeatmydata.so notmuch tag +test -- from:"sebastian@sspaeth.de"

real	0m1.286s
user	0m1.230s
sys	0m0.010s
---------------------------------------------------

Re: please eat my data!

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

On Mon, 12 Apr 2010 15:33:35 +0200, "Sebastian Spaeth" <Sebastian@SSpaeth.de> wrote:
> fsync is really killing xapian (and notmuch). What suffers, are the
> boolean prefixes (tag, id, and thread). Using libeatmydata (which
> disables fsync) shows a 10x speedup for tagging. The speedup is only
> factor 2 for e.g. from: searches. This is ext4 on recent stock
> Ubuntu. Given that search by tag and thread are performed really often
> (each time I advance a thread, for example), that really hurts.

Wow, this is really interesting, Sebastian.  For those of us not in the
know, can you explain what libeatmydata is and how it's used?  It sounds
like something I would *not* want to use!  So you didn't have to
recompile here, and only had to set LD_PRELOAD=./libeatmydata.so?  Is
there any drawback to what you're doing here?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: please eat my data!

[racin]

----- "Jameson Rollins" <jrollins@finestructure.net> a écrit :

> On Mon, 12 Apr 2010 15:33:35 +0200, "Sebastian Spaeth"
> <Sebastian@SSpaeth.de> wrote:
> > fsync is really killing xapian (and notmuch). What suffers, are the
> > boolean prefixes (tag, id, and thread). Using libeatmydata (which
> > disables fsync) shows a 10x speedup for tagging. The speedup is
> only
> > factor 2 for e.g. from: searches. This is ext4 on recent stock
> > Ubuntu. Given that search by tag and thread are performed really
> often
> > (each time I advance a thread, for example), that really hurts.
> 
> Wow, this is really interesting, Sebastian.  For those of us not in
> the
> know, can you explain what libeatmydata is and how it's used?  It
> sounds
> like something I would *not* want to use!  So you didn't have to
> recompile here, and only had to set LD_PRELOAD=./libeatmydata.so?  Is
> there any drawback to what you're doing here?
> 
> jamie.
> 

It seems like it is a small library that implements fsync as no-op. Using LD_PRELOAD 
allows to overloads the libc's fsync definition by libeatmydata's one. Making writes faster,
but no longer crash-safe.

Matthieu

Re: please eat my data!

[Sebastian Spaeth]

On 2010-04-12, Jameson Rollins wrote:
> On Mon, 12 Apr 2010 15:33:35 +0200, "Sebastian Spaeth" > Wow, this is really interesting, Sebastian.  For those of us not in the
> know, can you explain what libeatmydata is and how it's used?

Hehe, I just got the pointer to it on IRC myself:

http://www.flamingspork.com/projects/libeatmydata/

You download and untar the thing, and "make" it, which produces
libeatmydata.so. Running a binary foo with LD_PRELOAD=./libeatmydata.so foo
will then effectively make all fsyncs a Noop. Not something you want on
your production systems, but great to test how much of a penality those
fsyncs really are.

What I find intersting is that we have a 2x speedup and a 10x speedup
for different queries. Olly was saying on IRC that both *should* really be
behaving in much the same manner.

Sebastian

Re: please eat my data!

[Stewart Smith]

On Mon, 12 Apr 2010 17:24:35 +0200, "Sebastian Spaeth" <Sebastian@SSpaeth.de> wrote:
> What I find intersting is that we have a 2x speedup and a 10x speedup
> for different queries. Olly was saying on IRC that both *should* really be
> behaving in much the same manner.

Remember that on ext3 (and pretty sure ext4) fsync is the same as
sync(). So performance depends on how much dirty data you have in your cache.

libeatmydata also gets rid of msync(), O_SYNC etc as well.

-- 
Stewart Smith

Re: please eat my data!

[Dirk Hohndel]

On Mon, 12 Apr 2010 10:14:05 -0700, Stewart Smith <stewart@flamingspork.com> wrote:
> On Mon, 12 Apr 2010 17:24:35 +0200, "Sebastian Spaeth" <Sebastian@SSpaeth.de> wrote:
> > What I find intersting is that we have a 2x speedup and a 10x speedup
> > for different queries. Olly was saying on IRC that both *should* really be
> > behaving in much the same manner.
> 
> Remember that on ext3 (and pretty sure ext4) fsync is the same as
> sync(). So performance depends on how much dirty data you have in your cache.
> 
> libeatmydata also gets rid of msync(), O_SYNC etc as well.

Which is why so many of us have started to use BTRFS...

Much smaller performance degradation when doing frequent fsync's

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center

Re: please eat my data!

[Servilio Afre Puentes]

On 12 April 2010 13:47, Dirk Hohndel <hohndel@infradead.org> wrote:
> On Mon, 12 Apr 2010 10:14:05 -0700, Stewart Smith <stewart@flamingspork.com> wrote:
>> On Mon, 12 Apr 2010 17:24:35 +0200, "Sebastian Spaeth" <Sebastian@SSpaeth.de> wrote:
>> > What I find intersting is that we have a 2x speedup and a 10x speedup
>> > for different queries. Olly was saying on IRC that both *should* really be
>> > behaving in much the same manner.
>>
>> Remember that on ext3 (and pretty sure ext4) fsync is the same as
>> sync(). So performance depends on how much dirty data you have in your cache.
>>
>> libeatmydata also gets rid of msync(), O_SYNC etc as well.
>
> Which is why so many of us have started to use BTRFS...

How stable is it now? What kernel version and distro are you using?

Servilio

Re: please eat my data!

[Dirk Hohndel]

On Mon, 12 Apr 2010 19:10:25 -0400, Servilio Afre Puentes <servilio@gmail.com> wrote:
> On 12 April 2010 13:47, Dirk Hohndel <hohndel@infradead.org> wrote:
> > On Mon, 12 Apr 2010 10:14:05 -0700, Stewart Smith <stewart@flamingspork.com> wrote:
> >> On Mon, 12 Apr 2010 17:24:35 +0200, "Sebastian Spaeth" <Sebastian@SSpaeth.de> wrote:
> >> > What I find intersting is that we have a 2x speedup and a 10x speedup
> >> > for different queries. Olly was saying on IRC that both *should* really be
> >> > behaving in much the same manner.
> >>
> >> Remember that on ext3 (and pretty sure ext4) fsync is the same as
> >> sync(). So performance depends on how much dirty data you have in your cache.
> >>
> >> libeatmydata also gets rid of msync(), O_SYNC etc as well.
> >
> > Which is why so many of us have started to use BTRFS...
> 
> How stable is it now? What kernel version and distro are you using?

Several. Fedora 12 with 2.6.34-rc3. Moblin-2.1 (derivative) with 2.6.33.
Debian sid with 2.6.33

I've been using it for most everything I do since some point in the
2.6.32 rcs.

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center