From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <tomi.ollila@iki.fi>
Received: from localhost (localhost [127.0.0.1])
 by arlo.cworth.org (Postfix) with ESMTP id 89FC86DE2027
 for <notmuch@notmuchmail.org>; Tue, 21 Feb 2017 12:26:01 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at cworth.org
X-Spam-Flag: NO
X-Spam-Score: 0.507
X-Spam-Level: 
X-Spam-Status: No, score=0.507 tagged_above=-999 required=5 tests=[AWL=-0.145, 
 SPF_NEUTRAL=0.652] autolearn=disabled
Received: from arlo.cworth.org ([127.0.0.1])
 by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id eFH0EEwHHQ7H for <notmuch@notmuchmail.org>;
 Tue, 21 Feb 2017 12:26:00 -0800 (PST)
Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])
 by arlo.cworth.org (Postfix) with ESMTP id C35DC6DE2025
 for <notmuch@notmuchmail.org>; Tue, 21 Feb 2017 12:25:59 -0800 (PST)
Received: from guru.guru-group.fi (localhost [IPv6:::1])
 by guru.guru-group.fi (Postfix) with ESMTP id 1ADA91001A7;
 Tue, 21 Feb 2017 22:25:25 +0200 (EET)
From: Tomi Ollila <tomi.ollila@iki.fi>
To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org
Subject: Re: read after free in notmuch new
In-Reply-To: <87d1ec5ki4.fsf@tethera.net>
References: <87efyu6zdg.fsf@tethera.net> <87bmty6vwu.fsf@tethera.net>
 <87d1ec5ki4.fsf@tethera.net>
User-Agent: Notmuch/0.23.3+85~g2b85e66 (https://notmuchmail.org) Emacs/24.5.1
 (x86_64-unknown-linux-gnu)
X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL
 $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F
 !)g;OY^,BjTbr)Np:%c_o'jj,Z
Date: Tue, 21 Feb 2017 22:25:24 +0200
Message-ID: <m2tw7ne1fv.fsf@guru.guru-group.fi>
MIME-Version: 1.0
Content-Type: text/plain
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "Use and development of the notmuch mail system."
 <notmuch.notmuchmail.org>
List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,
 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch/>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,
 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Tue, 21 Feb 2017 20:26:01 -0000

On Tue, Feb 21 2017, David Bremner <david@tethera.net> wrote:

> David Bremner <david@tethera.net> writes:
>
>> David Bremner <david@tethera.net> writes:
>>
>>> I haven't had a chance to really track this down, but it seems there is
>>> a memory error in notmuch new (or a maybe false positive from valgrind).
>>>
>>> Attached is the log from running "make memory-test OPTIONS=--medium" on
>>> current git master (0e037c34).
>>>
>>> It looks like we talloc the message_id string with the message object as
>>> parent, but it somehow outlives the message object.
>>
>> Sorry, that had a few commits beyond master.
>>
>> master (08343d3d) gives essentially the same log.
>>
>
> The log says the relevent piece of memory was freed at line 655 of database.cc, which
> is the g_hash_table_insert in the code 
>
> 	ref = _parse_message_id (ctx, refs, &refs);
>
> 	if (ref && strcmp (ref, message_id)) {
> 	    g_hash_table_insert (hash, ref, NULL);
> 	    last_ref = ref;
> 	}
>
>
> According to the docs for g_hash_table_insert
>
>    If the key already exists in the GHashTable its current value is
>    replaced with the new value. If you supplied a value_destroy_func
>    when creating the GHashTable, the old value is freed using that
>    function. If you supplied a key_destroy_func when creating the
>    GHashTable, the passed key is freed using that function.
>
> Since we do pass a key_destroy_func, it seems we are being naughty by
> returning last_ref just below.

To me it looks like replacing g_hash_table_insert() with 
g_hash_table_replace() would do the trick.

(or even g_hash_table_add()!)

One has to read the documentation a bit (and compare the docstrings of
these 2 functions to guess the missing pieces) to get some understanding to
this...

Tomi