From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 8D+SC4Mzvl+LUAAA0tVLHw (envelope-from ) for ; Wed, 25 Nov 2020 10:35:47 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id sOJgB4Mzvl8jBQAAbx9fmQ (envelope-from ) for ; Wed, 25 Nov 2020 10:35:47 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2F4AA9403CA for ; Wed, 25 Nov 2020 10:35:45 +0000 (UTC) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id CDC7E28CA4; Wed, 25 Nov 2020 05:35:35 -0500 (EST) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by mail.notmuchmail.org (Postfix) with ESMTPS id 8734B28C9E for ; Wed, 25 Nov 2020 05:35:32 -0500 (EST) Received: by mail-wr1-x42b.google.com with SMTP id i2so1393419wrs.4 for ; Wed, 25 Nov 2020 02:35:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:content-transfer-encoding:subject:from:to:in-reply-to :references:date:message-id:user-agent; bh=kwkEN3bBXKZbL//OLA6spU0xkGN8KPT3jVnasSAbcko=; b=ptuiMd1ChRsXCDffUCNwsAGkndn3El6rHmizuoMlLCu9MXbFQjEUZXjq4sC0+W86yO 2eA8G40oG2ATIG7nEkllNt/6zouJrlN0WI7j3Y1soZqAriA1KB33REZ0oxhLB1aIhsx6 lMig1IKllaOTn10JWx9yRInVcIwpFkbeJhRrvMtXbsyY4tq8wClzRVI8RK3pdWSEl7O+ i2sKLTeNQLTwrMvGYPqXWBl+rqB/F4i8pRt1KwV4V1xOoTxrmsEUjSEqsuW73ziR/nub KTgUxOCrAbGXm5PonSNpxywU0fK1qHagRe0zmDuBp6qhZBM5Lcp7aFu7pSVZrf7Klc7n 4dUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:content-transfer-encoding:subject :from:to:in-reply-to:references:date:message-id:user-agent; bh=kwkEN3bBXKZbL//OLA6spU0xkGN8KPT3jVnasSAbcko=; b=M8vT6bjunyZ0GeNzlEY/hr/XnCZU4VCvhNLs8FyMbv3FKYcAb2h+uQ9Xn1eK+yH9r4 ea67p7Z3IIlzxGQZheL3X7b8ArgC5Yx7mWq/BC7Hbrv8VVYGaztftpE2bdsezo5fWkKs xQEv+xlsBwLSLjB3PZCIh+4kXafgWZsEM+L9xhchGrpuNYvyTJs+jwUDv8ceCNqglcQq 657y2fApBso3E1aOctRqp5wmouCB/L44n1vy19xiY/CcUUVDcm1PxvlTnAXLlaNbedr9 CKgGXWOt8QVImnj31hX3eftJgpqYETB4kiYea8DH1seBjNkLit41IpdqCdHGF+BnnZNC kNUw== X-Gm-Message-State: AOAM533wMi4uJZyORndhi2AIpiJWneizTx3yAggN9SfELj3fVXolAXJR yuR5LiMTFFZ+fA3DdfHVIUM= X-Google-Smtp-Source: ABdhPJwWYYF3xQmcd60fxaqeCIC08Fvl/VV96mj7/gfbJnL/Kq45VEnt6FCgCq89ZEhuXn03MkpHsQ== X-Received: by 2002:a5d:44c1:: with SMTP id z1mr3213066wrr.375.1606300529805; Wed, 25 Nov 2020 02:35:29 -0800 (PST) Received: from localhost (2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4.3.5.d.1.c.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7ac1:d534::2]) by smtp.gmail.com with ESMTPSA id b14sm3950386wrq.47.2020.11.25.02.35.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Nov 2020 02:35:29 -0800 (PST) MIME-Version: 1.0 Subject: Re: notmuch2 (python cffi bindings) segfault gdb logs From: Patrick Totzke To: Floris Bruynooghe , notmuch@notmuchmail.org In-Reply-To: <871rgiqw6j.fsf@powell.devork.be> References: <160612776000.3630981.17332396369796619495@piu> <871rgiqw6j.fsf@powell.devork.be> Date: Wed, 25 Nov 2020 10:35:27 +0000 Message-ID: <160630052747.135120.6464979361290025771@piu> User-Agent: alot/0.9.1 Message-ID-Hash: XPCUMUFS2UL2N3JM4AHDKWOTRUDQDZSG X-Message-ID-Hash: XPCUMUFS2UL2N3JM4AHDKWOTRUDQDZSG X-MailFrom: patricktotzke@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=ptuiMd1C; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Spam-Score: 0.59 X-TUID: hgtK6sKk/Js0 Hello Floris, thanks for having a look at this! Quoting Floris Bruynooghe (2020-11-24 21:31:00) > Hi Patrick, > > On Mon 23 Nov 2020 at 10:36 +0000, Patrick Totzke wrote: > > I've been complaining about the new (and old) python bindings causing the python interpreter to segfault occasionally. So far I was not able to reproduce this reliably nor provide error traces. This has just changed: > > see below and attached for what I got from gdb. > > Your gdb info doesn't say explicitly (or I missed it), but this is > showing a SEGFAULT I guess? Yes, correct. I saw this triggered when untagging some messages from my inbox in alot. I forgot to mention version numbers: notmuch: 0.31+7~g981d5a0 Python: 3.8.6 alot: 0.9.1 notmuch and bindings are compiled from git master, on a debian testing system. > > I hope that whoever is in charge of the bindings can make sense of > > it. I don't have any experience so far with cffi nor gdb and have a > > hard time debugging this. The logs below are my attempt to collect as > > much detail as possible about. Please let me know if I missed > > something. > > From what I can tell we're calling a function to free something which > segfaults, so it probably was freed already and we didn't know. We need > to find out who freed it before and why we thought it still needed to be > freed. It may help to know that this only ever happened if i tagged messages while the alot screen did not display the whole query result. I presume that this means there was some left over reference to an existing query object, which could have been affected by libtalloc. Alot is reading thread id's from notmuch2.Database.threads() in a generator: https://github.com/pazz/alot/blob/master/alot/db/manager.py#L314 Could this be problematic? After all, it may continue reading from it after a while. > > (gdb) info threads > > Id Target Id Frame > > * 1 Thread 0x7ffff7c0e740 (LWP 3614451) "python3" __GI_raise (sig=sig@entry=6) > > at ../sysdeps/unix/sysv/linux/raise.c:50 > > From this I gather we only have one thread, could you confirm this? > notmuch2 just isn't thread safe at the moment (I forget whether this was > intentional or by accident, might have been intentional depending on how > threadsafe libnotmuch is). Yes, I'm quite -- but not 100% --- sure as I did not write the port to notmuch2 for alot's backend. > > Traceback (most recent call first): > > > > File "/home/pazz/.local/lib/python3.8/site-packages/notmuch2/_thread.py", line 38, in _destroy > > capi.lib.notmuch_thread_destroy(self._thread_p) > > File "/home/pazz/.local/lib/python3.8/site-packages/notmuch2/_thread.py", line 34, in __del__ > > self._destroy() > > File "/home/pazz/projects/alot/alot/db/manager.py", line 570, in get_threads > > > > I pulled alot master and this does not match at all. Could you tell me > which git ref this was using so I can try and see what alot is actually > doing? (or some other way of sharing the source in this backtrace) This happed on alot master: 7915ea60ba866010abc728851626df96d8b80816 for me. I should say that I've had this issue long before, even before alot used the new bindings. Another stab in the dark: Could this be due to concurrent changes to the notmuch index in my mail sync/tagging script? I am using afew https://github.com/afewmail/afew which is still on the old python bindings as far as I am aware. Thanks again for your efforts Floris! P