From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id SyLKJ0G3mmCs/QAAgWs5BA (envelope-from ) for ; Tue, 11 May 2021 18:56:33 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id GE+yIkG3mmD2OwAAB5/wlQ (envelope-from ) for ; Tue, 11 May 2021 16:56:33 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 369681048B for ; Tue, 11 May 2021 18:56:33 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 9C87427180; Tue, 11 May 2021 12:56:25 -0400 (EDT) Received: from mailproxy06.manitu.net (mailproxy06.manitu.net [217.11.48.70]) by mail.notmuchmail.org (Postfix) with ESMTPS id 1027E20017 for ; Tue, 11 May 2021 12:56:23 -0400 (EDT) Received: from localhost (200116b86017d900ae2f9e0a625a9927.dip.versatel-1u1.de [IPv6:2001:16b8:6017:d900:ae2f:9e0a:625a:9927]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: michael@grubix.eu) by mailproxy06.manitu.net (Postfix) with ESMTPSA id 7B10B5800AE; Tue, 11 May 2021 18:56:09 +0200 (CEST) MIME-Version: 1.0 In-Reply-To: <162074768557.189534.10699410442728726615.git@grubix.eu> References: <162020111427.14194.6147957596067303106.git@grubix.eu> <87lf8tl5xp.fsf@tethera.net> <162048871025.4412.16478596244616302794.git@grubix.eu> <17c04a657363f96fac1c477f238b96d6@condition-alpha.com> <9a30dccb8d00c3de861abb3057331949@condition-alpha.com> <162074768557.189534.10699410442728726615.git@grubix.eu> Subject: Re: Possible threading issues in nm 0.32 From: Michael J Gruber To: Alexander Adolf , David Bremner , notmuch@notmuchmail.org Message-ID: <162075216869.247978.11973039285204032566.git@grubix.eu> Date: Tue, 11 May 2021 18:56:08 +0200 User-Agent: alot/0.9.1 Message-ID-Hash: IJY24F6Z7B6IJHAIPHPUC4IQ5IVSCEHB X-Message-ID-Hash: IJY24F6Z7B6IJHAIPHPUC4IQ5IVSCEHB X-MailFrom: michael@grubix.eu X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1620752193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=a5xCNLEdh3iOtpRqcyYBzE/hvjjml0GWP7tF4j9hm80=; b=izCPMsjlvtjT2NM/c2fh1wiOIvCrzJ0StzyhFRIykfTuoX3NObYgrofmQ1A4AJtHCIeb6V GPgKTzyK9yTUOPODKomDDoA8w6piX2u+SQX6hWEnA9kYcWPRFlSwq0aAJYNTsGW67e4Xb9 79aq3mygqfTnqkASrFmCY4UgdHyKH2VzPlAq5tVGG9tpUL6I4P9io1LOfUICnQsmBxLcTD r+7cCWKJxawUVLPzFomkcTsrX/D9/QLR/trP+b1bZ+5Mi9whS2jgix/GP4hZu2AcnitEOq YrAOLMtwDetxdqLi66jsa0zOy/Xg62c/fm4fO7Om2pfdkDUCUzsswGRWDFBa+Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1620752193; a=rsa-sha256; cv=none; b=U2JqMxGPDxqp+l5ut76+HbEe/lIuXdpaRLdf519gDP/4Sn55GTONRGCFS0ezgRdROoa4e0 aAJ7N0df30PQ9RxF4V7zxSJTI9yWcXTozIIzBrACqcwQLnnWRyPd3fML45faQ4Xpbjs+Uc BusUPRBzeyVJNL/x2sGlwPun3womIIEwSYBoX4uU0blDt/AMD3IF2itwxyO28f/B1qH0Do jpeqH/oZ7GC4/GGZQwvSZmd/Gb4mwkZA52FHgI5/Yr0itfhDN06cUxRIWrpSefif1/tlso TCuCI9d1h/JqxGNz+BR8m7UOd8URiq3tkDcKCOHG5P5zYhYlNCpvIGy0EUi8XA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 0.44 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 369681048B X-Spam-Score: 0.44 X-Migadu-Scanner: scn0.migadu.com X-TUID: Nw4l0mr+kAwC ... just a guess: Could it be that a9f74aee ("CLI/new: drop the write lock to run the pre-new hook.", 2021-03-18) was not enough? notmuch_database_reopen() only reopens the xapian db but does not update other members in notmuch_database_t *notmuch, such as the last doc id and thread id. If a pre-merge hook changes the database then values in that struct will be out of date. Before the config changes in nm 0.32, there was no such struct to begin with. After that, notmuch holds the struct just to be able to run the hook (from the proper dir). So I would think that reopen()ing to MODE_READ_ONLY is not a problem. But after the hook run, are full close() then open() to MODE_READ_WRITE() is necessary so that the values in the struct are correct (or change reopen() to do that). Indeed, if you open in MODE_READ_ONLY and don't hold a write lock you cannot trust any cached values such as those stored in the struct, can you? Michael