From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id qPk6ACM6tl5SSAAA0tVLHw (envelope-from ) for ; Sat, 09 May 2020 05:05:39 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id SAArNy86tl5vPwAA1q6Kng (envelope-from ) for ; Sat, 09 May 2020 05:05:51 +0000 Received: from arlo.cworth.org (arlo.cworth.org [50.126.95.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 62A0C940D8A for ; Sat, 9 May 2020 05:05:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 93E726DE0F7A; Fri, 8 May 2020 22:05:46 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s3MtHqtDQzsm; Fri, 8 May 2020 22:05:46 -0700 (PDT) Received: from arlo.cworth.org (localhost [IPv6:::1]) by arlo.cworth.org (Postfix) with ESMTP id EAE1C6DE0F5A; Fri, 8 May 2020 22:05:43 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 1C09E6DE0F5A for ; Fri, 8 May 2020 22:05:42 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y2f9pkggm3zm for ; Fri, 8 May 2020 22:05:40 -0700 (PDT) Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by arlo.cworth.org (Postfix) with ESMTPS id B90556DE0ABA for ; Fri, 8 May 2020 22:05:39 -0700 (PDT) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 3D98A289EF6 for ; Sat, 9 May 2020 07:05:38 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id z6-k7xV8Fcu3 for ; Sat, 9 May 2020 07:05:37 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 46403289E5F for ; Sat, 9 May 2020 07:05:37 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 113F22029B for ; Sat, 9 May 2020 07:05:37 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id UwTOkNoHuiaR for ; Sat, 9 May 2020 07:05:35 +0200 (CEST) Received: from dev.daenerys.khirnov.net (dev.daenerys.khirnov.net [IPv6:2a00:c500:561:201::6]) by quelana.khirnov.net (Postfix) with ESMTP id F313320228 for ; Sat, 9 May 2020 07:05:34 +0200 (CEST) Received: by dev.daenerys.khirnov.net (Postfix, from userid 1000) id BB4645C1A05; Sat, 9 May 2020 07:05:34 +0200 (CEST) From: Anton Khirnov To: notmuch@notmuchmail.org Subject: [PATCH 1/2] python/notmuch2: do not destroy messages owned by a query Date: Sat, 9 May 2020 07:05:25 +0200 Message-Id: <20200509050526.23148-1-anton@khirnov.net> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: notmuch-bounces@notmuchmail.org Sender: "notmuch" X-Scanner: scn0 X-Spam-Score: -0.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 50.126.95.6 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Scan-Result: default: False [-0.01 / 13.00]; GENERIC_REPUTATION(0.00)[-0.45421300272233]; DWL_DNSWL_BLOCKED(0.00)[50.126.95.6:from]; IP_REPUTATION_HAM(0.00)[asn: 27017(-0.18), country: US(-0.00), ip: 50.126.95.6(-0.45)]; R_SPF_ALLOW(-0.20)[+a]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWELVE(0.00)[13]; MX_GOOD(-0.50)[cached: notmuchmail.org]; MAILLIST(-0.20)[mailman]; RCVD_IN_DNSWL_FAIL(0.00)[50.126.95.6:server fail]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:27017, ipnet:50.126.64.0/18, country:US]; MIME_TRACE(0.00)[0:+]; FROM_NEQ_ENVFROM(0.00)[anton@khirnov.net,notmuch-bounces@notmuchmail.org]; ARC_NA(0.00)[]; URIBL_BLOCKED(0.00)[notmuchmail.org:email]; FROM_HAS_DN(0.00)[]; SPF_REPUTATION_HAM(0.00)[-0.43500358352008]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[notmuch@notmuchmail.org]; DMARC_NA(0.00)[khirnov.net]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; MID_CONTAINS_FROM(1.00)[]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: h0Mig1gLZvOo Any messages retrieved from a query - either directly via search_messages() or indirectly via thread objects - are owned by that query. Retrieving the same message (i.e. corresponding to the same message ID / database object) several times will always yield the same C object. The caller is allowed to destroy message objects owned by a query before the query itself - which can save memory for long-lived queries. However, that message must then never be retrieved again from that query. The python-notmuch2 bindings will currently destroy every message object in Message._destroy(), which will lead to an invalid free if the same message is then retrieved again. E.g. the following python program leads to libtalloc abort()ing: import notmuch2 db = notmuch2.Database(mode = notmuch2.Database.MODE.READ_ONLY) t = next(db.threads('*')) msgs = list(zip(t.toplevel(), t.toplevel())) msgs = list(zip(t.toplevel(), t.toplevel())) Fix this issue by creating a subclass of Message, which is used for "standalone" message which have to be freed by the caller. Message class is then used only for messages descended from a query, which do not need to be freed by the caller. --- bindings/python-cffi/notmuch2/_database.py | 6 ++-- bindings/python-cffi/notmuch2/_message.py | 42 ++++++++++++++-------- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/bindings/python-cffi/notmuch2/_database.py b/bindings/python-cffi/notmuch2/_database.py index 95f59ca0..f14eac78 100644 --- a/bindings/python-cffi/notmuch2/_database.py +++ b/bindings/python-cffi/notmuch2/_database.py @@ -399,7 +399,7 @@ class Database(base.NotmuchObject): capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID] if ret not in ok: raise errors.NotmuchError(ret) - msg = message.Message(self, msg_pp[0], db=self) + msg = message.StandaloneMessage(self, msg_pp[0], db=self) if sync_flags: msg.tags.from_maildir_flags() return self.AddedMessage( @@ -468,7 +468,7 @@ class Database(base.NotmuchObject): msg_p = msg_pp[0] if msg_p == capi.ffi.NULL: raise LookupError - msg = message.Message(self, msg_p, db=self) + msg = message.StandaloneMessage(self, msg_p, db=self) return msg def get(self, filename): @@ -501,7 +501,7 @@ class Database(base.NotmuchObject): msg_p = msg_pp[0] if msg_p == capi.ffi.NULL: raise LookupError - msg = message.Message(self, msg_p, db=self) + msg = message.StandaloneMessage(self, msg_p, db=self) return msg @property diff --git a/bindings/python-cffi/notmuch2/_message.py b/bindings/python-cffi/notmuch2/_message.py index c5fdbf6d..416ce7ca 100644 --- a/bindings/python-cffi/notmuch2/_message.py +++ b/bindings/python-cffi/notmuch2/_message.py @@ -14,7 +14,7 @@ __all__ = ['Message'] class Message(base.NotmuchObject): - """An email message stored in the notmuch database. + """An email message stored in the notmuch database retrieved via a query. This should not be directly created, instead it will be returned by calling methods on :class:`Database`. A message keeps a @@ -61,22 +61,10 @@ class Message(base.NotmuchObject): @property def alive(self): - if not self._parent.alive: - return False - try: - self._msg_p - except errors.ObjectDestroyedError: - return False - else: - return True - - def __del__(self): - self._destroy() + return self._parent.alive def _destroy(self): - if self.alive: - capi.lib.notmuch_message_destroy(self._msg_p) - self._msg_p = None + pass @property def messageid(self): @@ -375,6 +363,30 @@ class Message(base.NotmuchObject): if isinstance(other, self.__class__): return self.messageid == other.messageid +class StandaloneMessage(Message): + """An email message stored in the notmuch database. + + This subclass of Message is used for messages that are retrieved from the + database directly and are not owned by a query. + """ + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._msg_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def __del__(self): + self._destroy() + + def _destroy(self): + if self.alive: + capi.lib.notmuch_message_destroy(self._msg_p) + self._msg_p = None class FilenamesIter(base.NotmuchIter): """Iterator for binary filenames objects.""" -- 2.20.1