From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id eI0tNhEC+V7eYAAA0tVLHw (envelope-from ) for ; Sun, 28 Jun 2020 20:48:17 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 8KLWMREC+V7qSQAAbx9fmQ (envelope-from ) for ; Sun, 28 Jun 2020 20:48:17 +0000 Received: from arlo.cworth.org (arlo.cworth.org [50.126.95.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3E528940215 for ; Sun, 28 Jun 2020 20:48:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 01F526DE102E; Sun, 28 Jun 2020 13:48:08 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NMRySl5wRBeT; Sun, 28 Jun 2020 13:48:07 -0700 (PDT) Received: from arlo.cworth.org (localhost [IPv6:::1]) by arlo.cworth.org (Postfix) with ESMTP id C391D6DE13AF; Sun, 28 Jun 2020 13:48:06 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id EF3A36DE13AF for ; Sun, 28 Jun 2020 13:48:04 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Uu2AtBaZC1tN for ; Sun, 28 Jun 2020 13:48:01 -0700 (PDT) Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by arlo.cworth.org (Postfix) with ESMTPS id D5B146DE102E for ; Sun, 28 Jun 2020 13:48:00 -0700 (PDT) Received: by mail-ed1-f65.google.com with SMTP id d18so5523975edv.6 for ; Sun, 28 Jun 2020 13:48:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:in-reply-to:references:date:message-id :mime-version; bh=KSqBJFVmgCGPo/PxubTmGB1I6gmV+E8obZWRLH+6oOs=; b=VfZzMXwbDVUaQUxvJl4RbGdZxSKU/dk0Fx3BYviBgs0qeJ2Fsp0vkVhz2G1/cLozAG Uztv08sUFKJgWsS3G8Pl0zVqplefbA3EKMEe/r3NtBG9KwZdyD3YiasLOUjx5jCq/fk5 Xrwhb4PRJxzI0sl7Y5FBIu2BoTJVbcKSOMsVMEqEMqR+9o3yV2S/Uel5LUmK+YHr9bWQ jCsX+wBh0nOeRzgv7vKS3pR17heJDRTvogFyK9J2U74Mploz0bj/jVfejN5uOrMOEeHE tnNKk8A2fDN+cdEnXEjxJyKFGCA56yZp6anZsXzx3WdEMhmz9wSTR2pylhieMkplP4Yy RfmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:in-reply-to:references :date:message-id:mime-version; bh=KSqBJFVmgCGPo/PxubTmGB1I6gmV+E8obZWRLH+6oOs=; b=E64rZWrjNhQfM5KzYGcJ9n3xJMO8BphICt1+1hwgqdOq6xIaMfzhtDF53ZLyYdOHJV HbpF1WV4B7sZ2hMEuPGjB6vizl3GdDpvzrx8OA1ATy/zIKEyS256T3xeWEKit1nbg+fF Y4ljTSPzjoJglsw8k+jSza10vLoXwlKdxNZCSo14YG80ierPW01Gt9lMUrAbfsOOTMju tMH2NeZ9IBhiyAXt8sfLzndI2LlaV1waJJCvXTwBsJQ/594kBv4btO8JzENGw5fOy7Vi rnIWoJByGP8CtHK4JG8llh2OfMMozZn5UxLjVXmckHxJyzwXHEGpdmvD+SOz+UxmGtOe CHSQ== X-Gm-Message-State: AOAM531DxaDRY08IXxo3QAz7y3Q3UVDAMSsOFDVt9QwOdT324q8b0CLm AfSzdW1fKgjRbu4Jz65WQ6A= X-Google-Smtp-Source: ABdhPJyQigYjocqNjhtn0w3UeYlumbnmZcVBGXBvcLfJMAwUfoRqYucyPBGzrbdhzFNcQOVZm6wC4w== X-Received: by 2002:a05:6402:1c8f:: with SMTP id cy15mr14384587edb.308.1593377279054; Sun, 28 Jun 2020 13:47:59 -0700 (PDT) Received: from powell.devork.be (2a02-8388-8480-1180-4c18-fc69-8d8c-22b5.cable.dynamic.v6.surfer.at. [2a02:8388:8480:1180:4c18:fc69:8d8c:22b5]) by smtp.gmail.com with ESMTPSA id 23sm12712671edx.75.2020.06.28.13.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Jun 2020 13:47:58 -0700 (PDT) Received: (nullmailer pid 39420 invoked by uid 1000); Sun, 28 Jun 2020 20:47:57 -0000 From: Floris Bruynooghe To: David Bremner , notmuch@notmuchmail.org Subject: Re: Usage after database close In-Reply-To: <87d05jkvry.fsf@tethera.net> References: <87d05je1j6.fsf@powell.devork.be> <87d05jkvry.fsf@tethera.net> Date: Sun, 28 Jun 2020 22:47:57 +0200 Message-ID: <875zbazzlu.fsf@powell.devork.be> MIME-Version: 1.0 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: notmuch-bounces@notmuchmail.org Sender: "notmuch" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=VfZzMXwb; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 50.126.95.6 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Spam-Score: -0.01 X-TUID: Jc/MrkOhcQZD On Sun 28 Jun 2020 at 13:19 -0300, David Bremner wrote: > Floris Bruynooghe writes: > >> Hi, >> >> I started writing some test cases to define better what you can do with >> a closed database and make sure that the python bindings do not behave >> unexpectedly here too. >> >> One of the first things I tried ends up with xapian calling >> exit_group(2) directly, terminating the process. So I'm wondering if >> I'm approaching this entirely the wrong way. My understanding is that >> we should generally be allowed to use anything after the database has >> been closed, as long as nothing has been destroyed. >> >> Below is a minimal reproducible example of what I'm testing so far. I >> must admit I'm generally lazy here and usually just test with notmuch >> that is currently in Debian testing. > > Funny that you should mention lazy, that's basically what the problem is > here ;). notmuch_message_get_message_id is lazily trying to read the > information from the database. This is a bit surprising here because of > the query, but that's not really visible once the message object is > created. > > In principle it could be documented what parts of the API can trigger > access to the database, but I'm not sure about the benefit of the extra > complexity. It might be safer to assume that only access to already > fetched information is safe. In particular if you move > > messageid = notmuch_message_get_message_id(msg); > > before you close the database, then printing it out afterwards works. I > didn't run it valgrind to make sure, but afaik, that should be perfectly > legal. Ok, I forgot the "expected behaviour" part of the bug report ;) I think that this doesn't work is fine and I'm not surprised by and your description of fetching it first is very reasonable. However I was expecting NOTMUCH_STATUS_XAPIAN_EXEPTION instead of bluntly getting terminated. This is what the notmuch_database_close() docs say after all. I had a little look and this seems to be caused by the message->doc.termlist_begin() call in _notmuch_message_ensure_metadata(), I didn't have xapian debug symbols and am not familiar with xapian to quickly have an idea of whether this case can be improved or not. (-dbg debian packages for notmuch and xapian would be very handy ;)) But part of my question is, *should* this be improved? Am I interpreting notmuch's intended API correctly? > The original motivation (see 7864350c938944276c1a378539da7670c211b9b5) > to allow long running processes to release the lock on the > database. This is not a pattern we use in the CLI, so it's not as well > tested as it could be. In particular the work to export > notmuch_database_reopen (tests, documentation) has not happened yet. > > d