From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 5BBB76DE1725 for ; Wed, 15 Feb 2017 14:58:57 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.72 X-Spam-Level: X-Spam-Status: No, score=-0.72 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1k1m2otsoyXr for ; Wed, 15 Feb 2017 14:58:56 -0800 (PST) Received: from mail-lf0-f54.google.com (mail-lf0-f54.google.com [209.85.215.54]) by arlo.cworth.org (Postfix) with ESMTPS id 3C97B6DE1724 for ; Wed, 15 Feb 2017 14:58:56 -0800 (PST) Received: by mail-lf0-f54.google.com with SMTP id v186so358812lfa.1 for ; Wed, 15 Feb 2017 14:58:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gaute-vetsj-com.20150623.gappssmtp.com; s=20150623; h=date:from:subject:to:references:in-reply-to:user-agent:message-id :mime-version; bh=+QiUKR3EXh33W2H7ihia0on5fIPohoTzBvi1f/QoQ74=; b=f8RqJ4qaC5XF1vq9b9U9H4T1iEmA3JRD/zWCoKkLotdfQHjJ629SGhpoc0Goh7ymUf gna+4BAw02yW8IkXAFcTI9nr1rp0UN8+m+2neFMLP+3Zei3c0c2/bpulNwwvTTKHb+a+ +4m+e0TeQwQoMjgAQXpgB74hEwjlBJxsX/wK/8LVxcN55p+J14q2jYMrp1CcpZfLYK3Y U2y+4lOjc7Ko+aNR1fl4nl9/Dj31v4QATBSxBHu9pXYM578oy//IIkvoB2o/wm4m7r7E +w+QYXbH97wKmAAjEY7fH8515L9KxhDMxMq5UfX/TfAzLDMklLWGpoYMG4xnQ1NoJ2gQ 9jGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:references:in-reply-to :user-agent:message-id:mime-version; bh=+QiUKR3EXh33W2H7ihia0on5fIPohoTzBvi1f/QoQ74=; b=t0X0mS8DZJKEn02ugh5b5zCfL4AFSUWIlD9wBYxsg6Fgj17S1v+F0UNZF1vkVS14tc fe0EUk/+hEH9HNDOCX+yTfRJk2d+HS04JBtGUSIoq4qliHQ7AppNCmYokqz2Y/g6kmyL MQGFgyRzHcd5yBEDD/xD95GuSZTn/U0KYbccMrizTLAF9dqcSMVxprYW3K4ZD+jg5W1L Zq70arSCAqlb/l7jZGDk7Ycm9JcqHGSUYWdzG23KKqMKDsTs9LrLuAGVjpX+jfHV9MB4 Sv5rUqNVvj0oTtIJlSXLMlbMWT+a8tyYEul2G3LhNtLWFxxbMvAE+z2gRQjuE+Kd5HT4 +HTA== X-Gm-Message-State: AMke39kq7OHsFL/GUWqQTDjRBu7r0Pl2/+A+aWRgQSyQkuwZw0VFMKm9tck0Yi+ythogjg== X-Received: by 10.46.81.18 with SMTP id f18mr1801698ljb.136.1487199534030; Wed, 15 Feb 2017 14:58:54 -0800 (PST) Received: from localhost (cm-84.215.128.252.getinternet.no. [84.215.128.252]) by smtp.gmail.com with ESMTPSA id 5sm1288304lja.58.2017.02.15.14.58.52 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Feb 2017 14:58:52 -0800 (PST) Date: Wed, 15 Feb 2017 22:58:51 +0000 From: Gaute Hope Subject: Re: talloc_abort in notmuch_thread_get_tags () when db has been modified To: David Bremner , notmuch@notmuchmail.org References: <1453121100-astroid-2-c62fwcrm91-18877@strange> <1457341792-astroid-0-2wtydh6y1q-15951@strange> <87r3fmts2y.fsf@zancas.localnet> In-Reply-To: <87r3fmts2y.fsf@zancas.localnet> User-Agent: astroid/v0.7-12-gb0a7a3a2 (https://github.com/astroidmail/astroid) Message-Id: <1487199120.tz5yvag0pm.astroid@strange.none> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-ycgv+abOKy78CaMd/YLV" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Feb 2017 22:58:57 -0000 --=-ycgv+abOKy78CaMd/YLV Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable David Bremner writes on mars 7, 2016 13:01: > Gaute Hope writes: >=20 >> as far as I can see, there is _no_ way to catch this error without >> completely crashing the application. I would have to isolate this code >> in a separate process or trap SIGABRT (which is certainly messy). >=20 > I'm not sure what you expect libnotmuch to do here. There's a fatal > "should not happen" error in the memory allocator; it isn't really the > sort of thing one can recover from. It's also not in code we control. >=20 > Of course _why_ this error is happening could still be notmuch's > fault. Can you reproduce the problem under valgrind? Hi again, For future reference: Attached is C++ test code that demonstrates the probl= em (at least on my setup). It is part of the astroid test suite. The test-code must be adapted to your _test_ notmuch db. To pick up on this again, this issue started cropping up more frequently again, and I can't see a way currently to anticipate or recover from this from a user application of the notmuch library. There seems to be an XapianError, which may or may not be handled by notmuch. Regards, Gaute = --=-ycgv+abOKy78CaMd/YLV Content-Type: text/x-c++src; name=test_notmuch_standalone.cc Content-Disposition: attachment; filename=test_notmuch_standalone.cc Content-Transfer-Encoding: base64 IyBpbmNsdWRlIDxpb3N0cmVhbT4KIyBpbmNsdWRlIDxib29zdC9maWxlc3lzdGVtLmhwcD4KCiMg aW5jbHVkZSA8bm90bXVjaC5oPgoKbmFtZXNwYWNlIGJmcyA9IGJvb3N0OjpmaWxlc3lzdGVtOwp1 c2luZyBzdGQ6OmNvdXQ7CnVzaW5nIHN0ZDo6ZW5kbDsKCmludCBtYWluICgpIHsKICBiZnM6OnBh dGggcGF0aF9kYiA9IGJmczo6YWJzb2x1dGUgKGJmczo6cGF0aCgiLi90ZXN0L21haWwvdGVzdF9t YWlsIikpOwoKICBub3RtdWNoX2RhdGFiYXNlX3QgKiBubV9kYjsKCiAgbm90bXVjaF9zdGF0dXNf dCBzID0KICAgIG5vdG11Y2hfZGF0YWJhc2Vfb3BlbiAoCiAgICAgIHBhdGhfZGIuY19zdHIoKSwK ICAgICAgbm90bXVjaF9kYXRhYmFzZV9tb2RlX3Q6Ok5PVE1VQ0hfREFUQUJBU0VfTU9ERV9SRUFE X09OTFksCiAgICAgICZubV9kYik7CgoKICBjb3V0IDw8ICJkYjogcnVubmluZyB0ZXN0IHF1ZXJ5 Li4iIDw8IGVuZGw7CiAgbm90bXVjaF9xdWVyeV90ICogcSA9IG5vdG11Y2hfcXVlcnlfY3JlYXRl IChubV9kYiwgIioiKTsKCiAgdW5zaWduZWQgaW50IGM7CiAgbm90bXVjaF9zdGF0dXNfdCBzdCA9 IG5vdG11Y2hfcXVlcnlfY291bnRfdGhyZWFkc19zdCAocSwgJmMpOyAvLyBkZXN0cnVjdGl2ZQog IG5vdG11Y2hfcXVlcnlfZGVzdHJveSAocSk7CiAgcSA9IG5vdG11Y2hfcXVlcnlfY3JlYXRlIChu bV9kYiwgIioiKTsKCiAgY291dCA8PCAicXVlcnk6ICIgPDwgbm90bXVjaF9xdWVyeV9nZXRfcXVl cnlfc3RyaW5nIChxKSA8PCAiLCBhcHByb3g6ICIKICAgICAgIDw8IGMgPDwgIiB0aHJlYWRzLiIg PDwgZW5kbDsKCiAgbm90bXVjaF90aHJlYWRzX3QgKiB0aHJlYWRzOwogIG5vdG11Y2hfdGhyZWFk X3QgICogdGhyZWFkOwogIHN0ID0gbm90bXVjaF9xdWVyeV9zZWFyY2hfdGhyZWFkc19zdCAocSwg JnRocmVhZHMpOwoKICBzdGQ6OnN0cmluZyB0aHJlYWRfaWQ7CgogIGludCBpID0gMDsKCiAgZm9y ICg7IG5vdG11Y2hfdGhyZWFkc192YWxpZCAodGhyZWFkcyk7CiAgICAgICAgIG5vdG11Y2hfdGhy ZWFkc19tb3ZlX3RvX25leHQgKHRocmVhZHMpKSB7CiAgICB0aHJlYWQgPSBub3RtdWNoX3RocmVh ZHNfZ2V0ICh0aHJlYWRzKTsKICAgIGkrKzsKCiAgICBpZiAoaSA9PSAzKQogICAgICB0aHJlYWRf aWQgPSBub3RtdWNoX3RocmVhZF9nZXRfdGhyZWFkX2lkICh0aHJlYWQpOwoKICAgIG5vdG11Y2hf dGhyZWFkX2Rlc3Ryb3kgKHRocmVhZCk7CgogICAgaWYgKGkgPT0gMykgYnJlYWs7CiAgfQoKICBj b3V0IDw8ICJ0aHJlYWQgaWQgdG8gY2hhbmdlOiAiIDw8IHRocmVhZF9pZCA8PCAiLCB0aHJlYWQg bm86ICIgPDwgaSA8PCBlbmRsOwogIG5vdG11Y2hfcXVlcnlfZGVzdHJveSAocSk7CgogIC8qIHJl c3RhcnQgcXVlcnkgKi8KICBjb3V0IDw8ICJyZXN0YXJ0aW5nIHF1ZXJ5Li4iIDw8IGVuZGw7CiAg cSA9IG5vdG11Y2hfcXVlcnlfY3JlYXRlIChubV9kYiwgIioiKTsKICBzdCA9IG5vdG11Y2hfcXVl cnlfc2VhcmNoX3RocmVhZHNfc3QgKHEsICZ0aHJlYWRzKTsKCiAgaSA9IDA7CiAgaW50IHN0b3Ag PSAyOwoKICBjb3V0IDw8ICJtb3ZpbmcgdG8gdGhyZWFkOiAiIDw8IHN0b3AgPDwgZW5kbDsKICBm b3IgKCA7IG5vdG11Y2hfdGhyZWFkc192YWxpZCAodGhyZWFkcyk7CiAgICAgICAgICBub3RtdWNo X3RocmVhZHNfbW92ZV90b19uZXh0ICh0aHJlYWRzKSkKICB7CiAgICB0aHJlYWQgPSBub3RtdWNo X3RocmVhZHNfZ2V0ICh0aHJlYWRzKTsKICAgIG5vdG11Y2hfdGhyZWFkX2dldF90aHJlYWRfaWQg KHRocmVhZCk7CiAgICBpKys7CgogICAgY291dCA8PCAidGFnczogIjsKCiAgICAvKiBnZXQgdGFn cyAqLwogICAgbm90bXVjaF90YWdzX3QgKnRhZ3M7CiAgICBjb25zdCBjaGFyICp0YWc7CgogICAg Zm9yICh0YWdzID0gbm90bXVjaF90aHJlYWRfZ2V0X3RhZ3MgKHRocmVhZCk7CiAgICAgICAgIG5v dG11Y2hfdGFnc192YWxpZCAodGFncyk7CiAgICAgICAgIG5vdG11Y2hfdGFnc19tb3ZlX3RvX25l eHQgKHRhZ3MpKQogICAgewogICAgICAgIHRhZyA9IG5vdG11Y2hfdGFnc19nZXQgKHRhZ3MpOwog ICAgICAgIGNvdXQgPDwgdGFnIDw8ICIgIjsKICAgIH0KICAgIGNvdXQgPDwgZW5kbDsKCiAgICBu b3RtdWNoX3RocmVhZF9kZXN0cm95ICh0aHJlYWQpOwoKICAgIGlmIChpID09IHN0b3ApIGJyZWFr OwogIH0KCiAgLyogbm93IG9wZW4gYSBuZXcgZGIgaW5zdGFuY2UsIG1vZGlmeSB0aGUgYWxyZWFk eSBsb2FkZWQgdGhyZWFkIGFuZAogICAqIGNvbnRpbnVlIGxvYWRpbmcgdGhlIG9yaWdpbmFsIHF1 ZXJ5ICovCiAgbm90bXVjaF9kYXRhYmFzZV90ICogbm1fZGIyOwoKICBzID0gbm90bXVjaF9kYXRh YmFzZV9vcGVuICgKICAgICAgcGF0aF9kYi5jX3N0cigpLAogICAgICBub3RtdWNoX2RhdGFiYXNl X21vZGVfdDo6Tk9UTVVDSF9EQVRBQkFTRV9NT0RFX1JFQURfV1JJVEUsCiAgICAgICZubV9kYjIp OwoKCiAgY2hhciBxcnlfc1syNTZdOwogIHNwcmludGYgKHFyeV9zLCAidGhyZWFkOiVzIiwgdGhy ZWFkX2lkLmNfc3RyICgpKTsKICBub3RtdWNoX3F1ZXJ5X3QgKiBxMiA9IG5vdG11Y2hfcXVlcnlf Y3JlYXRlIChubV9kYjIsIHFyeV9zKTsKICBub3RtdWNoX3RocmVhZHNfdCAqIHRzMjsKICBub3Rt dWNoX3RocmVhZF90ICAqIHQyOwoKICBzdCA9IG5vdG11Y2hfcXVlcnlfc2VhcmNoX3RocmVhZHNf c3QgKHEyLCAmdHMyKTsKCiAgZm9yICggOyBub3RtdWNoX3RocmVhZHNfdmFsaWQgKHRzMik7CiAg ICAgICAgICBub3RtdWNoX3RocmVhZHNfbW92ZV90b19uZXh0ICh0czIpKQogIHsKICAgIHQyID0g bm90bXVjaF90aHJlYWRzX2dldCAodHMyKTsKICAgIHN0ZDo6c3RyaW5nIHRocmVhZF9pZCA9IG5v dG11Y2hfdGhyZWFkX2dldF90aHJlYWRfaWQgKHQyKTsKCgogICAgLyogcmVtb3ZlIHVucmVhZCB0 YWcgKi8KICAgIG5vdG11Y2hfbWVzc2FnZXNfdCAqIG1zID0gbm90bXVjaF90aHJlYWRfZ2V0X21l c3NhZ2VzICh0Mik7CiAgICBub3RtdWNoX21lc3NhZ2VfdCAgKiBtOwoKICAgIGZvciAoOyBub3Rt dWNoX21lc3NhZ2VzX3ZhbGlkIChtcyk7IG5vdG11Y2hfbWVzc2FnZXNfbW92ZV90b19uZXh0ICht cykpIHsKICAgICAgbSA9IG5vdG11Y2hfbWVzc2FnZXNfZ2V0IChtcyk7CgogICAgICBzdCA9IG5v dG11Y2hfbWVzc2FnZV9yZW1vdmVfdGFnIChtLCAidW5yZWFkIik7CgogICAgICBub3RtdWNoX21l c3NhZ2VfZGVzdHJveSAobSk7CiAgICB9CgogICAgbm90bXVjaF9tZXNzYWdlc19kZXN0cm95ICht cyk7CgoKICAgIG5vdG11Y2hfdGhyZWFkX2Rlc3Ryb3kgKHQyKTsKICAgIGJyZWFrOwogIH0KCiAg bm90bXVjaF9xdWVyeV9kZXN0cm95IChxMik7CiAgbm90bXVjaF9kYXRhYmFzZV9jbG9zZSAobm1f ZGIyKTsKCiAgLyogcmUtYWRkIHVucmVhZCB0YWcgKi8KICBzID0gbm90bXVjaF9kYXRhYmFzZV9v cGVuICgKICAgICAgcGF0aF9kYi5jX3N0cigpLAogICAgICBub3RtdWNoX2RhdGFiYXNlX21vZGVf dDo6Tk9UTVVDSF9EQVRBQkFTRV9NT0RFX1JFQURfV1JJVEUsCiAgICAgICZubV9kYjIpOwoKCgog IHEyID0gbm90bXVjaF9xdWVyeV9jcmVhdGUgKG5tX2RiMiwgcXJ5X3MpOwoKICBzdCA9IG5vdG11 Y2hfcXVlcnlfc2VhcmNoX3RocmVhZHNfc3QgKHEyLCAmdHMyKTsKCiAgZm9yICggOyBub3RtdWNo X3RocmVhZHNfdmFsaWQgKHRzMik7CiAgICAgICAgICBub3RtdWNoX3RocmVhZHNfbW92ZV90b19u ZXh0ICh0czIpKQogIHsKICAgIHQyID0gbm90bXVjaF90aHJlYWRzX2dldCAodHMyKTsKICAgIHN0 ZDo6c3RyaW5nIHRocmVhZF9pZCA9IG5vdG11Y2hfdGhyZWFkX2dldF90aHJlYWRfaWQgKHQyKTsK CgogICAgLyogcmVtb3ZlIHVucmVhZCB0YWcgKi8KICAgIG5vdG11Y2hfbWVzc2FnZXNfdCAqIG1z ID0gbm90bXVjaF90aHJlYWRfZ2V0X21lc3NhZ2VzICh0Mik7CiAgICBub3RtdWNoX21lc3NhZ2Vf dCAgKiBtOwoKICAgIGZvciAoOyBub3RtdWNoX21lc3NhZ2VzX3ZhbGlkIChtcyk7IG5vdG11Y2hf bWVzc2FnZXNfbW92ZV90b19uZXh0IChtcykpIHsKICAgICAgbSA9IG5vdG11Y2hfbWVzc2FnZXNf Z2V0IChtcyk7CgogICAgICBzdCA9IG5vdG11Y2hfbWVzc2FnZV9hZGRfdGFnIChtLCAidW5yZWFk Iik7CgogICAgICBub3RtdWNoX21lc3NhZ2VfZGVzdHJveSAobSk7CiAgICB9CgogICAgbm90bXVj aF9tZXNzYWdlc19kZXN0cm95IChtcyk7CgoKICAgIG5vdG11Y2hfdGhyZWFkX2Rlc3Ryb3kgKHQy KTsKICAgIGJyZWFrOwogIH0KCiAgbm90bXVjaF9xdWVyeV9kZXN0cm95IChxMik7CiAgbm90bXVj aF9kYXRhYmFzZV9jbG9zZSAobm1fZGIyKTsKCiAgLyogY29udGludWUgbG9hZGluZyAqLwogIGNv dXQgPDwgImNvbnRpbnVlIGxvYWRpbmcuLiIgPDwgZW5kbDsKICBmb3IgKCA7IG5vdG11Y2hfdGhy ZWFkc192YWxpZCAodGhyZWFkcyk7CiAgICAgICAgICBub3RtdWNoX3RocmVhZHNfbW92ZV90b19u ZXh0ICh0aHJlYWRzKSkKICB7CiAgICBpZiAodGhyZWFkcyA9PSBOVUxMKSB7CiAgICAgIGNvdXQg PDwgInRocmVhZHMgPT0gTlVMTCIgPDwgZW5kbDsKICAgIH0gZWxzZSB7CiAgICAgIGNvdXQgPDwg InRocmVhZHMgIT0gTlVMTCIgPDwgZW5kbDsKICAgIH0KICAgIHRocmVhZCA9IG5vdG11Y2hfdGhy ZWFkc19nZXQgKHRocmVhZHMpOwogICAgY291dCA8PCAibG9hZGluZzogIiA8PCBpOwogICAgc3Rk OjpzdHJpbmcgdGlkID0gbm90bXVjaF90aHJlYWRfZ2V0X3RocmVhZF9pZCAodGhyZWFkKTsKICAg IGNvdXQgPDwgIjogIiA8PCB0aWQgPDwgZW5kbDsKCiAgICAvKiBnZXQgdGFncyAqLwogICAgbm90 bXVjaF90YWdzX3QgKnRhZ3M7CiAgICBjb25zdCBjaGFyICp0YWc7CgogICAgY291dCA8PCAidGFn czogIjsKICAgIGZvciAodGFncyA9IG5vdG11Y2hfdGhyZWFkX2dldF90YWdzICh0aHJlYWQpOwog ICAgICAgICBub3RtdWNoX3RhZ3NfdmFsaWQgKHRhZ3MpOwogICAgICAgICBub3RtdWNoX3RhZ3Nf bW92ZV90b19uZXh0ICh0YWdzKSkKICAgIHsKICAgICAgICB0YWcgPSBub3RtdWNoX3RhZ3NfZ2V0 ICh0YWdzKTsKICAgICAgICBjb3V0IDw8IHRhZyA8PCAiICI7CiAgICB9CiAgICBjb3V0IDw8IGVu ZGw7CgogICAgaSsrOwogICAgbm90bXVjaF90aHJlYWRfZGVzdHJveSAodGhyZWFkKTsKICB9Cgog IG5vdG11Y2hfZGF0YWJhc2VfY2xvc2UgKG5tX2RiKTsKICByZXR1cm4gMDsKfQoK --=-ycgv+abOKy78CaMd/YLV--