From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id gEZELAau0WGkjQAAgWs5BA (envelope-from ) for ; Sun, 02 Jan 2022 14:52:06 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id oJavKAau0WERQQAAauVa8A (envelope-from ) for ; Sun, 02 Jan 2022 14:52:06 +0100 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 25F80F35C for ; Sun, 2 Jan 2022 14:52:06 +0100 (CET) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 08F6C5F6ED; Sun, 2 Jan 2022 13:52:01 +0000 (UTC) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by mail.notmuchmail.org (Postfix) with ESMTP id DFD4D5F478 for ; Sun, 2 Jan 2022 13:51:57 +0000 (UTC) Received: by fethera.tethera.net (Postfix, from userid 1001) id CB08B5FC42; Sun, 2 Jan 2022 08:51:56 -0500 (EST) Received: (nullmailer pid 571137 invoked by uid 1000); Sun, 02 Jan 2022 13:51:55 -0000 From: David Bremner To: notmuch@notmuchmail.org Subject: use after free in python notmuch2 bindings X-List-To: notmuch Date: Sun, 02 Jan 2022 09:51:55 -0400 Message-ID: <87sfu6utxg.fsf@tethera.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Message-ID-Hash: TOTJ4LQ2SKHYAXJHUQ7QOT6PJATDSAVJ X-Message-ID-Hash: TOTJ4LQ2SKHYAXJHUQ7QOT6PJATDSAVJ X-MailFrom: david@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Migadu-Flow: FLOW_IN X-Migadu-Country: DE ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641131526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type:list-id: list-help:list-owner:list-unsubscribe:list-subscribe:list-post; bh=ATVP5RxrxDGORysc68+InYUUEHJ28DBOd/F7UG9q2Hk=; b=uaImBVft5uJliJOjtnAQbz0LkQ9y//vVz3URi2IC/0O/Jgih/+9PiUpFTjXQnsKepkkw2L 9Coo+MnRURuGUyaqBJicVm1Rlk9fbxBpmhjXTWk9883KrAoJ5uN7rN4mEaqBU8iTWW4t8V +71CmiUAZxsgiCl53lkLbXWB/OkqMYqYT1lXxool0MW/OKANQXV0u9m/7rJ7R+Ywu1pTPt jyjxueEEJaiiR2cawthWl8NxfjAXIQVBHES5UFwXszR1my5jkuGN3rVD21vLlyTDBgR1ys yce8aU7s3MLpaqa8gbOQ+i1QEHzn2zxvJtxxCJWS+4c7K0QWkk+9dTGPBuKO1Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641131526; a=rsa-sha256; cv=none; b=Q4O5yOX2XOaOxmEU2PVtQRfjLqNMoIcInKiCZ4LUbIzW37RLNyTEOUqT6Dm9S+hZuITaw/ 4KIwyA8ZgvNbtZFW0Z38aHN2m+eNbrWjfD54VdoXzoNrNg4aCofAlQxEdi0yaY9kR6miWP a+SXF9OhrUgkgLUfBieREwRVXu6Zlvzdkz3FGnnKyOICRVfwtVFlYYaJv74BPdrx/lSqnl POqkEx4uHxoLHI37lbyOq02dG/KW65DqpGIlI7Ub0LzZV7bRJz8u9dp7bDv7wCLE55qEPR Ax4Q7nhs5W9jXv2uq82XRc7z2wDlkIAfPqevuSbaKNKiI2U+YSbvy6ugIMRjZQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -2.28 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 25F80F35C X-Spam-Score: -2.28 X-Migadu-Scanner: scn1.migadu.com X-TUID: v1O8yAST3uyC --=-=-= Content-Type: text/plain I've been attempting to port nmweb to the new bindings, but I got stuck on a bug that segfaults python. I attached a reduced version that reproduces the problem for me. It uses recent messages from the notmuch list; it others can't reproduce let me know and I will try to make something more self contained including a message set. It's a bit tricky to get ASAN working but I managed with % env ASAN_OPTIONS=alloc_dealloc_mismatch=0 LD_PRELOAD="libasan.so.6 libstdc++.so.6" LD_LIBRARY_PATH=../../lib python3 ~/test.py You can see in the attached output that one of the notmuch messages structs is used after being freed. I suspect it has something to do with the iterator code in the bindings, but I have not examined it in detail. --=-=-= Content-Type: text/x-python Content-Disposition: inline; filename=test.py Content-Description: reproducer for use after free from notmuch2 import Database def mailto_addrs(msg,header_name): hdr = msg.header(header_name) return def show_msgs(msgs): print("show msgs" + str(msgs)) for msg in msgs: print("\t",msg.messageid) frm = mailto_addrs(msg,'From') rs = show_msgs(msg.replies()) return db = Database(config=Database.CONFIG.SEARCH) msg=db.find("87fsqijx7u.fsf@metapensiero.it") threads = db.threads(query="thread:"+msg.threadid) thread = next (threads) show_msgs(thread) --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=asan.out Content-Transfer-Encoding: base64 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT0KPT01NzEwODc9PUVSUk9SOiBBZGRyZXNzU2FuaXRpemVyOiBoZWFwLXVzZS1hZnRl ci1mcmVlIG9uIGFkZHJlc3MgMHg2MTIwMDAwMDIxYjAgYXQgcGMgMHg3ZjI1MDBkMzY2NzEgYnAg MHg3ZmZkYjlmYTg2NzAgc3AgMHg3ZmZkYjlmYTg2NjgKUkVBRCBvZiBzaXplIDggYXQgMHg2MTIw MDAwMDIxYjAgdGhyZWFkIFQwCiAgICAjMCAweDdmMjUwMGQzNjY3MCBpbiBub3RtdWNoX21lc3Nh Z2VfZ2V0X21lc3NhZ2VfaWQgbGliL21lc3NhZ2UuY2M6NTI1CiAgICAjMSAweDdmMjUwMTYxMmI3 NiBpbiBfY2ZmaV9mX25vdG11Y2hfbWVzc2FnZV9nZXRfbWVzc2FnZV9pZCBidWlsZC90ZW1wLmxp bnV4LXg4Nl82NC0zLjkvbm90bXVjaDIuX2NhcGkuYzozMDc5CiAgICAjMiAweDUyNGRiYyAgKC91 c3IvYmluL3B5dGhvbjMuOSsweDUyNGRiYykKICAgICMzIDB4NTE0NjUzIGluIF9QeUV2YWxfRXZh bEZyYW1lRGVmYXVsdCAoL3Vzci9iaW4vcHl0aG9uMy45KzB4NTE0NjUzKQogICAgIzQgMHg1MjU5 MTIgaW4gX1B5RnVuY3Rpb25fVmVjdG9yY2FsbCAoL3Vzci9iaW4vcHl0aG9uMy45KzB4NTI1OTEy KQogICAgIzUgMHg1MzQyYjIgICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MzQyYjIpCiAgICAjNiAw eDUyM2Y1NyBpbiBfUHlPYmplY3RfR2VuZXJpY0dldEF0dHJXaXRoRGljdCAoL3Vzci9iaW4vcHl0 aG9uMy45KzB4NTIzZjU3KQogICAgIzcgMHg1MGY2MzQgaW4gX1B5RXZhbF9FdmFsRnJhbWVEZWZh dWx0ICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MGY2MzQpCiAgICAjOCAweDUyNTkxMiBpbiBfUHlG dW5jdGlvbl9WZWN0b3JjYWxsICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MjU5MTIpCiAgICAjOSAw eDUwZjcxZSBpbiBfUHlFdmFsX0V2YWxGcmFtZURlZmF1bHQgKC91c3IvYmluL3B5dGhvbjMuOSsw eDUwZjcxZSkKICAgICMxMCAweDUyNTkxMiBpbiBfUHlGdW5jdGlvbl9WZWN0b3JjYWxsICgvdXNy L2Jpbi9weXRob24zLjkrMHg1MjU5MTIpCiAgICAjMTEgMHg1MGY3MWUgaW4gX1B5RXZhbF9FdmFs RnJhbWVEZWZhdWx0ICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MGY3MWUpCiAgICAjMTIgMHg1MGRl YjAgICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MGRlYjApCiAgICAjMTMgMHg1MGRjMjYgaW4gX1B5 RXZhbF9FdmFsQ29kZVdpdGhOYW1lICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MGRjMjYpCiAgICAj MTQgMHg1MGRiZDIgaW4gUHlFdmFsX0V2YWxDb2RlICgvdXNyL2Jpbi9weXRob24zLjkrMHg1MGRi ZDIpCiAgICAjMTUgMHg2MjliYzYgICgvdXNyL2Jpbi9weXRob24zLjkrMHg2MjliYzYpCiAgICAj MTYgMHg2MjZiNmYgICgvdXNyL2Jpbi9weXRob24zLjkrMHg2MjZiNmYpCiAgICAjMTcgMHg2Mjk1 ZTggICgvdXNyL2Jpbi9weXRob24zLjkrMHg2Mjk1ZTgpCiAgICAjMTggMHg2MjkyNjIgaW4gUHlS dW5fU2ltcGxlRmlsZUV4RmxhZ3MgKC91c3IvYmluL3B5dGhvbjMuOSsweDYyOTI2MikKICAgICMx OSAweDYyMGQ2MSBpbiBQeV9SdW5NYWluICgvdXNyL2Jpbi9weXRob24zLjkrMHg2MjBkNjEpCiAg ICAjMjAgMHg2MDhiZjggaW4gUHlfQnl0ZXNNYWluICgvdXNyL2Jpbi9weXRob24zLjkrMHg2MDhi ZjgpCiAgICAjMjEgMHg3ZjI1MDQ2ZTA3ZWMgaW4gX19saWJjX3N0YXJ0X21haW4gLi4vY3N1L2xp YmMtc3RhcnQuYzozMzIKICAgICMyMiAweDYwOGFmOSBpbiBfc3RhcnQgKC91c3IvYmluL3B5dGhv bjMuOSsweDYwOGFmOSkKCjB4NjEyMDAwMDAyMWIwIGlzIGxvY2F0ZWQgMTEyIGJ5dGVzIGluc2lk ZSBvZiAyNzItYnl0ZSByZWdpb24gWzB4NjEyMDAwMDAyMTQwLDB4NjEyMDAwMDAyMjUwKQpmcmVl ZCBieSB0aHJlYWQgVDAgaGVyZToKICAgICMwIDB4N2YyNTA0Y2ZiNGQ3IGluIF9faW50ZXJjZXB0 b3JfZnJlZSAuLi8uLi8uLi8uLi9zcmMvbGlic2FuaXRpemVyL2FzYW4vYXNhbl9tYWxsb2NfbGlu dXguY3BwOjEyNwogICAgIzEgMHg3ZjI1MDE0MTEzNTMgaW4gX3RjX2ZyZWVfaW50ZXJuYWwgLi4v Li4vdGFsbG9jLmM6MTIyMgoKcHJldmlvdXNseSBhbGxvY2F0ZWQgYnkgdGhyZWFkIFQwIGhlcmU6 CiAgICAjMCAweDdmMjUwNGNmYjdjZiBpbiBfX2ludGVyY2VwdG9yX21hbGxvYyAuLi8uLi8uLi8u Li9zcmMvbGlic2FuaXRpemVyL2FzYW4vYXNhbl9tYWxsb2NfbGludXguY3BwOjE0NQogICAgIzEg MHg3ZjI1MDE0MTM3ZWYgaW4gX190YWxsb2Nfd2l0aF9wcmVmaXggLi4vLi4vdGFsbG9jLmM6Nzgz CgpTVU1NQVJZOiBBZGRyZXNzU2FuaXRpemVyOiBoZWFwLXVzZS1hZnRlci1mcmVlIGxpYi9tZXNz YWdlLmNjOjUyNSBpbiBub3RtdWNoX21lc3NhZ2VfZ2V0X21lc3NhZ2VfaWQKU2hhZG93IGJ5dGVz IGFyb3VuZCB0aGUgYnVnZ3kgYWRkcmVzczoKICAweDBjMjQ3ZmZmODNlMDogZmQgZmQgZmQgZmQg ZmQgZmQgZmQgZmQgZmQgZmQgZmEgZmEgZmEgZmEgZmEgZmEKICAweDBjMjQ3ZmZmODNmMDogZmEg ZmEgZmEgZmEgZmEgZmEgZmEgZmEgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAKICAweDBjMjQ3ZmZm ODQwMDogMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAKICAw eDBjMjQ3ZmZmODQxMDogMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgZmEgZmEgZmEgZmEg ZmEgZmEKICAweDBjMjQ3ZmZmODQyMDogZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmQgZmQgZmQg ZmQgZmQgZmQgZmQgZmQKPT4weDBjMjQ3ZmZmODQzMDogZmQgZmQgZmQgZmQgZmQgZmRbZmRdZmQg ZmQgZmQgZmQgZmQgZmQgZmQgZmQgZmQKICAweDBjMjQ3ZmZmODQ0MDogZmQgZmQgZmQgZmQgZmQg ZmQgZmQgZmQgZmQgZmQgZmEgZmEgZmEgZmEgZmEgZmEKICAweDBjMjQ3ZmZmODQ1MDogZmEgZmEg ZmEgZmEgZmEgZmEgZmEgZmEgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAKICAweDBjMjQ3ZmZmODQ2 MDogMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAKICAweDBj MjQ3ZmZmODQ3MDogMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgZmEgZmEgZmEgZmEgZmEg ZmEKICAweDBjMjQ3ZmZmODQ4MDogZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmEgZmEg ZmEgZmEgZmEgZmEKU2hhZG93IGJ5dGUgbGVnZW5kIChvbmUgc2hhZG93IGJ5dGUgcmVwcmVzZW50 cyA4IGFwcGxpY2F0aW9uIGJ5dGVzKToKICBBZGRyZXNzYWJsZTogICAgICAgICAgIDAwCiAgUGFy dGlhbGx5IGFkZHJlc3NhYmxlOiAwMSAwMiAwMyAwNCAwNSAwNiAwNyAKICBIZWFwIGxlZnQgcmVk em9uZTogICAgICAgZmEKICBGcmVlZCBoZWFwIHJlZ2lvbjogICAgICAgZmQKICBTdGFjayBsZWZ0 IHJlZHpvbmU6ICAgICAgZjEKICBTdGFjayBtaWQgcmVkem9uZTogICAgICAgZjIKICBTdGFjayBy aWdodCByZWR6b25lOiAgICAgZjMKICBTdGFjayBhZnRlciByZXR1cm46ICAgICAgZjUKICBTdGFj ayB1c2UgYWZ0ZXIgc2NvcGU6ICAgZjgKICBHbG9iYWwgcmVkem9uZTogICAgICAgICAgZjkKICBH bG9iYWwgaW5pdCBvcmRlcjogICAgICAgZjYKICBQb2lzb25lZCBieSB1c2VyOiAgICAgICAgZjcK ICBDb250YWluZXIgb3ZlcmZsb3c6ICAgICAgZmMKICBBcnJheSBjb29raWU6ICAgICAgICAgICAg YWMKICBJbnRyYSBvYmplY3QgcmVkem9uZTogICAgYmIKICBBU2FuIGludGVybmFsOiAgICAgICAg ICAgZmUKICBMZWZ0IGFsbG9jYSByZWR6b25lOiAgICAgY2EKICBSaWdodCBhbGxvY2EgcmVkem9u ZTogICAgY2IKICBTaGFkb3cgZ2FwOiAgICAgICAgICAgICAgY2MKPT01NzEwODc9PUFCT1JUSU5H Cg== --=-=-= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --=-=-=--