From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ACHTEMA0Q2EuhQEAgWs5BA (envelope-from ) for ; Thu, 16 Sep 2021 14:12:48 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id KCZzDMA0Q2HjMQAAB5/wlQ (envelope-from ) for ; Thu, 16 Sep 2021 12:12:48 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0012A17549 for ; Thu, 16 Sep 2021 14:12:47 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 98DE026C36; Thu, 16 Sep 2021 08:12:43 -0400 (EDT) X-Greylist: delayed 574 seconds by postgrey-1.36 at nmbug; Thu, 16 Sep 2021 08:12:41 EDT Received: from mailproxy04.manitu.net (mailproxy04.manitu.net [217.11.48.68]) by mail.notmuchmail.org (Postfix) with ESMTPS id 1E2EE26A98 for ; Thu, 16 Sep 2021 08:12:41 -0400 (EDT) Received: from localhost (200116b86053f30071ef3627e15627a1.dip.versatel-1u1.de [IPv6:2001:16b8:6053:f300:71ef:3627:e156:27a1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: michael@grubix.eu) by mailproxy04.manitu.net (Postfix) with ESMTPSA id 6E8D380450; Thu, 16 Sep 2021 14:03:05 +0200 (CEST) MIME-Version: 1.0 In-Reply-To: <20210916102517.1744389-1-andreas@rammhold.de> References: <20210916102517.1744389-1-andreas@rammhold.de> Subject: Re: [RFC 0/2] add --new-tags= to notmuch-new(1) To: andreas@rammhold.de, notmuch@notmuchmail.org From: Michael J Gruber Message-ID: <163179378465.773443.12112139662488774678.git@grubix.eu> Date: Thu, 16 Sep 2021 14:03:04 +0200 User-Agent: alot/0.9.1 Message-ID-Hash: XNYYFCLBNN3PBWB6KDV6ZINP5JVBVAGP X-Message-ID-Hash: XNYYFCLBNN3PBWB6KDV6ZINP5JVBVAGP X-MailFrom: michael@grubix.eu X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631794368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=dstRSzkvMvTHVY7nvCLLVh94m8oUSdcOPk/u7B95p7Q=; b=GIvzsMhcIVtBD/mgXQAytK4EXWyw5a+++8S7NlC7noYsSUI+OaL8GoxgGJmNuR+NMg93Ot QnKSwd6/4bU5CRvXYxWNa6J9g2+Bcgng494Zy2E4wdjot2Y6+lrWrB/qyQFwbktIsyE2tG zAUO0J4ffe2D3JNE2RxCkVXOHb3mJFOgya5ahGTVj6hCLoVSOKEUJxAu4xR6FvKHWV3KTB 6dlaBGSBqIMlF64MmOf2N4Q/2FXoB63n+m1gcgB/mtJ6iDG/8rNOR/bdhZfcYMO9v84MRb JddoD6QSL/yThtaRASR509lV/Us2C+w6pzkoaCDvcVLBkYESj92u1k18UHRHdw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631794368; a=rsa-sha256; cv=none; b=o7/I9LZxYeIQZRjntb1SJVgeSUxVuPOyReOgZqlAcC7zYvCtIc59NI2uqRwB5ahpqUZdxJ aPWZDbpGSAxQ4ngXx+rm4XB9FyfdvJz7xKXKdl39fmZtt7e6mKLaTvXCaI4jcil9qKZZpU SDpR2b0RK2vgVTyv+/voRCwqRNdRWNJGSJvf8lylwlrapVW4Mkei4Q1DjuzEcdv9eJINEQ cGNssJq02c0ZSuBgLZdO3y+3LoEyGHoAaJXcW33w4eSRorDJa7mV/ov6Q+wqG6LCgU49Ca bnIj8puIJvcw5O38ORdcHZ+e9NB9bpYZzXmqZCT8hIQ+RyfmHed6Mj7TISPosQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 0.39 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 0012A17549 X-Spam-Score: 0.39 X-Migadu-Scanner: scn1.migadu.com X-TUID: MHk+GWIHd/Iq andreas@rammhold.de venit, vidit, dixit 2021-09-16 12:25:15: > I've recently became a little annoyed with a race condition in my current > notmuch setup that originates from only having a single set of new tags for > all notmuch-new(1) invocations. In the past I've mentioned this a couple of > times in the IRC channel and now I got around to implement a basic version of > this. > > I rougly process new messages like this: > > 1. new messages get the "new" tag through notmuch-new(1), > 2. the post-new hook calls a series of scripts ... > There are right now 8 mailboxes that I am retrieving mails from. I have a > scheduled job that updates all my local Maildir's every couple of minutes. > That one doesn't cause any issues on its own. > > But I also use IMAP IDLE to selectively update Maildir's as soon as a new mail > arrives. > > If I receive mails on multiple Maildir's within a short period of time the > above process is running into race-conditions since there is no way to > distinguish mails that have just been marked new. All of them carry the same > tags. > > In the worst case one of the last steps (2c/2d) would pick up the new mail > before any of the actual classification has been executed. This "leaks" mails > into my inbox which consequently can be overwhelming to look at. > > With this series I am able to give each notmuch-new(1) invocation a unique tag > (think: new-$(uuidgen)). This doesn't (on its own) solve the entire story but > is a first step in the "right direction" IMHO. I still have to wrap the entire I very much sympathize with your setup. But I think the real solution would be one of these options: - use a lock file to prevent your scripts from running concurrently OR - match "tag:new and folder:UNIQUEFOLDER" (or "path:FOO/**") This should be a perfect substitute for your new-UNIQUE tag. As for the implementation you suggest: Basically, you implement overriding the "new.tags" config, and I'm wondering whether it would be worthwhile to implement "notmuch --config-value" instead: --config-value=SECTION.ITEM=VALUE Override the config setting for SECTION.ITEM with the VALUE for this invocation. This takes precedence over any setting in the config file. I know this raises the question which config any hooks will see, but that is the same for your implementation. You can override the complete config by specifying a different file already, of course, so you could script that. Cheers Michael