From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gHGVBSrh514UawAA0tVLHw (envelope-from ) for ; Mon, 15 Jun 2020 20:59:22 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id SFl7ASrh517bbQAAB5/wlQ (envelope-from ) for ; Mon, 15 Jun 2020 20:59:22 +0000 Received: from arlo.cworth.org (arlo.cworth.org [50.126.95.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 807D49401CF for ; Mon, 15 Jun 2020 20:59:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 671736DE0A77; Mon, 15 Jun 2020 13:59:19 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GklgC7UgL85W; Mon, 15 Jun 2020 13:59:18 -0700 (PDT) Received: from arlo.cworth.org (localhost [IPv6:::1]) by arlo.cworth.org (Postfix) with ESMTP id 1C5106DE0BB9; Mon, 15 Jun 2020 13:59:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 6E24C6DE0BB9 for ; Mon, 15 Jun 2020 13:59:16 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V5YqtM3Z2YcC for ; Mon, 15 Jun 2020 13:59:15 -0700 (PDT) Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by arlo.cworth.org (Postfix) with ESMTPS id 2F8326DE0A77 for ; Mon, 15 Jun 2020 13:59:15 -0700 (PDT) Received: by mail-ej1-f52.google.com with SMTP id n24so18989510ejd.0 for ; Mon, 15 Jun 2020 13:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vIri8WCXXA2bIhFwj8XWesG8DMaanLwL1Ej/LpXWl0k=; b=C+7Red3ONRABLXcOrPqKj0y5gZVNavpSmS5Nz4yOak2eIuIbkyjZpaghGwL0MVfDf0 GOSI8uMb2Vp2RCpzeMHc5dbcLExUAQuvx/qs3k6lEAZ43cKuxG7Bb/7lANiGrVTYEyBN z0EohWtZUw2xkOJ7TQVL9x+Lwmvopu7KegB0p6NWbEAIHk0+o1aQWrVQx48HG2/ptXRw zqPLNH63ejgKDFD2BLXrFnaFFqQlmjZh6AcFtiUUYb7pV06H87V9N8CZUXjRFprnrXIi VGv0MjQIG2aYfoyJ5Lx2qAa6DWQ7uDeaBW76ZzW7UYtIL+9rhDRyHpavkhJGsky0Ir/n pYXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=vIri8WCXXA2bIhFwj8XWesG8DMaanLwL1Ej/LpXWl0k=; b=Ffo1pUzWLFtMOYOKBnIcTrDOFKdTMUc5Ya4nNow3CXm1neJPcVVSfseQ0gEIq7X3Mj hedKGPrVE20377biQG6+71H6iUk3VxyasEtG3k6cra2tJOR/NbOfYinEFSbX0+p+sxqZ DqlFf+UFn8rMFCaWwEWUGLPAXzZ2s31ebbs/AH3k4Nj45rZw4zYYw+cN2EtKnI6XhN3m OqFJ4Z3aOuSZvJS+75E7Amwcv3wswfbtBv67ER/g0Ooi+ni5QAU8Z99E17KK7vqW4OHW hftubXSNNM9mIry00l2N37L54NNWcOZ9VpZgsztWzKWM8xl4Qena1/3mMmb0iocwYzcY VPng== X-Gm-Message-State: AOAM53323peOFnhBMFIVt/LS9kAJXvMvcHWr2YM51HrR5LugOFjjJFph 5D3sBfddwg90J1EBE6D2A8IxdxvO X-Google-Smtp-Source: ABdhPJxAL0OIPX+SX3trghftrQ2xFNtUF9k8iqNXvxPI0VzytUeA3QQ/xaFWH15bnwyP/L2yMY6/Gg== X-Received: by 2002:a17:906:b88c:: with SMTP id hb12mr26795973ejb.483.1592254753485; Mon, 15 Jun 2020 13:59:13 -0700 (PDT) Received: from powell.devork.be (2a02-8388-8480-1180-4c18-fc69-8d8c-22b5.cable.dynamic.v6.surfer.at. [2a02:8388:8480:1180:4c18:fc69:8d8c:22b5]) by smtp.gmail.com with ESMTPSA id h9sm9744295ejc.96.2020.06.15.13.59.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2020 13:59:13 -0700 (PDT) Received: (nullmailer pid 210686 invoked by uid 1000); Mon, 15 Jun 2020 20:59:12 -0000 From: Floris Bruynooghe To: notmuch@notmuchmail.org Subject: [PATCH 1/2] python/notmuch2: do not destroy messages owned by a query Date: Mon, 15 Jun 2020 22:58:49 +0200 Message-Id: <20200615205850.210480-2-flub@devork.be> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200615205850.210480-1-flub@devork.be> References: <20200615205850.210480-1-flub@devork.be> MIME-Version: 1.0 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: notmuch-bounces@notmuchmail.org Sender: "notmuch" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=C+7Red3O; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 50.126.95.6 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Spam-Score: 2.49 X-TUID: N4GJZahtK9Jn From: Anton Khirnov Any messages retrieved from a query - either directly via search_messages() or indirectly via thread objects - are owned by that query. Retrieving the same message (i.e. corresponding to the same message ID / database object) several times will always yield the same C object. The caller is allowed to destroy message objects owned by a query before the query itself - which can save memory for long-lived queries. However, that message must then never be retrieved again from that query. The python-notmuch2 bindings will currently destroy every message object in Message._destroy(), which will lead to an invalid free if the same message is then retrieved again. E.g. the following python program leads to libtalloc abort()ing: import notmuch2 db = notmuch2.Database(mode = notmuch2.Database.MODE.READ_ONLY) t = next(db.threads('*')) msgs = list(zip(t.toplevel(), t.toplevel())) msgs = list(zip(t.toplevel(), t.toplevel())) Fix this issue by creating a subclass of Message, which is used for "standalone" message which have to be freed by the caller. Message class is then used only for messages descended from a query, which do not need to be freed by the caller. --- bindings/python-cffi/notmuch2/_database.py | 6 ++-- bindings/python-cffi/notmuch2/_message.py | 42 ++++++++++++++-------- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/bindings/python-cffi/notmuch2/_database.py b/bindings/python-cffi/notmuch2/_database.py index 95f59ca0..f14eac78 100644 --- a/bindings/python-cffi/notmuch2/_database.py +++ b/bindings/python-cffi/notmuch2/_database.py @@ -399,7 +399,7 @@ class Database(base.NotmuchObject): capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID] if ret not in ok: raise errors.NotmuchError(ret) - msg = message.Message(self, msg_pp[0], db=self) + msg = message.StandaloneMessage(self, msg_pp[0], db=self) if sync_flags: msg.tags.from_maildir_flags() return self.AddedMessage( @@ -468,7 +468,7 @@ class Database(base.NotmuchObject): msg_p = msg_pp[0] if msg_p == capi.ffi.NULL: raise LookupError - msg = message.Message(self, msg_p, db=self) + msg = message.StandaloneMessage(self, msg_p, db=self) return msg def get(self, filename): @@ -501,7 +501,7 @@ class Database(base.NotmuchObject): msg_p = msg_pp[0] if msg_p == capi.ffi.NULL: raise LookupError - msg = message.Message(self, msg_p, db=self) + msg = message.StandaloneMessage(self, msg_p, db=self) return msg @property diff --git a/bindings/python-cffi/notmuch2/_message.py b/bindings/python-cffi/notmuch2/_message.py index c5fdbf6d..416ce7ca 100644 --- a/bindings/python-cffi/notmuch2/_message.py +++ b/bindings/python-cffi/notmuch2/_message.py @@ -14,7 +14,7 @@ __all__ = ['Message'] class Message(base.NotmuchObject): - """An email message stored in the notmuch database. + """An email message stored in the notmuch database retrieved via a query. This should not be directly created, instead it will be returned by calling methods on :class:`Database`. A message keeps a @@ -61,22 +61,10 @@ class Message(base.NotmuchObject): @property def alive(self): - if not self._parent.alive: - return False - try: - self._msg_p - except errors.ObjectDestroyedError: - return False - else: - return True - - def __del__(self): - self._destroy() + return self._parent.alive def _destroy(self): - if self.alive: - capi.lib.notmuch_message_destroy(self._msg_p) - self._msg_p = None + pass @property def messageid(self): @@ -375,6 +363,30 @@ class Message(base.NotmuchObject): if isinstance(other, self.__class__): return self.messageid == other.messageid +class StandaloneMessage(Message): + """An email message stored in the notmuch database. + + This subclass of Message is used for messages that are retrieved from the + database directly and are not owned by a query. + """ + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._msg_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def __del__(self): + self._destroy() + + def _destroy(self): + if self.alive: + capi.lib.notmuch_message_destroy(self._msg_p) + self._msg_p = None class FilenamesIter(base.NotmuchIter): """Iterator for binary filenames objects.""" -- 2.27.0