From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id CmEuGOMuQ2GAfAEAgWs5BA (envelope-from ) for ; Thu, 16 Sep 2021 13:47:47 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uA5JE+MuQ2HHJQAAB5/wlQ (envelope-from ) for ; Thu, 16 Sep 2021 11:47:47 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B70F22B294 for ; Thu, 16 Sep 2021 13:47:46 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id EFA4026871; Thu, 16 Sep 2021 07:39:42 -0400 (EDT) Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by mail.notmuchmail.org (Postfix) with ESMTPS id CB92320646 for ; Thu, 16 Sep 2021 06:25:20 -0400 (EDT) Received: by mail-ed1-x533.google.com with SMTP id q3so14897789edt.5 for ; Thu, 16 Sep 2021 03:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rammhold-de.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=LEiBBtUSd3Rl/VRSc9RvE4JWRoUVpa4hFQZZIpfjQZE=; b=Lzyml5snHLyOb9XuJLCsOzyTDWTPlmbBVrenf1Pmi4zWM3LHGrbkDsGjdUFUFyPv5Q myRRcx4kYsS+UZogZVgQKwQYV+KaUKiafvinwfW0AIrk/wCQ+D40hz9d1VCmluivsCvc zXyYRFNkqc9m5mENsWNJbOwxR+IashWyMSTQ0/GkbbryG4Sj4ERiOwuc6xZphq4Gre6Q CYP42iNCC/4FZySiSnMYKUvj34usXARvAkKQrhlahywY54VwWQ35uEvxDNIL0xNxoJd3 xCsAopexmJgADZUom7mevvSaN2SAIwmBkMZcIN81r1tJns6tNIt37rFCLiz/cNOU7Vz6 q9Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=LEiBBtUSd3Rl/VRSc9RvE4JWRoUVpa4hFQZZIpfjQZE=; b=Unt+zAObD9Njg5vneNp9jVbCs4KcvBZauQzzY0KzFU+2QDoHxUrx2OU/X8lacNmuzz OY7s7EZ94uRldtjkGSB93MD4QFsUxKgA4xfLfo47TfillzfviNJCBgpL3obs2oOKOvnD Z9vknl1InLnjqhFNq7S9/XyaAsueZXUWL/SjqH9DcBWWnWulVCx3CfyUuro7HeEDBBVl hpkQZSx4MsxRH+bOsekMamszzuxbV3Vf/G6ZLHWYvMHmZ5q3XepkuiLYSRdoEJmeV3rX rqQtCO3G/THBn7Vcld/+frwp+blNh2jyLQiJl/2V0lGAhqEu4VyBr0rrTnbhZ4HNBa9R Fxxw== X-Gm-Message-State: AOAM533wIo9S9fBWgzD26d/NMoWGZBHPg+kWwG5L6G8AEZ4mpz6bkxYU JxgrJmJVP8yIf6tnLzQ9q21rUM1CBvkUFA== X-Google-Smtp-Source: ABdhPJxH7uQfxcGOgA/xjc/Y+F2p4THO2ujgDLFQJN5aBlgDgROYpw7+c4UxaFElxoEcKmRbsxSXfQ== X-Received: by 2002:aa7:d895:: with SMTP id u21mr5604139edq.300.1631787918558; Thu, 16 Sep 2021 03:25:18 -0700 (PDT) Received: from localhost ([2a00:e67:5c9:a:2e15:c474:2ef7:bc26]) by smtp.gmail.com with ESMTPSA id q14sm1007977ejc.93.2021.09.16.03.25.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 03:25:17 -0700 (PDT) From: andreas@rammhold.de To: notmuch@notmuchmail.org Subject: [RFC 0/2] add --new-tags= to notmuch-new(1) Date: Thu, 16 Sep 2021 12:25:15 +0200 Message-Id: <20210916102517.1744389-1-andreas@rammhold.de> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 X-MailFrom: andreas@rammhold.de X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0 Message-ID-Hash: RKMXBZQRCFO2LXO34GGUXP5ZB2GHLMHQ X-Message-ID-Hash: RKMXBZQRCFO2LXO34GGUXP5ZB2GHLMHQ X-Mailman-Approved-At: Thu, 16 Sep 2021 11:39:40 -0400 X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631792866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=n/gTWOERjq++wkrov16VhfY6Aif8FcAVTtizyqPf3VQ=; b=RY7La+Z83/nC6YMAKC+lFOjlQfzk4gO7IU10Rxu04KmASnM9khd8/zekfe6oZkzniBwb0e LvjnxQhNMp6K0dAlh4Db0YkOAz/gynVOKKzvWQrS+zwOaX34uE9L1xCmJcD6wBz+RZkpmV iBUaspQvGYP6/TU9MkA2aDV14j4Gwxi0Hr5gTb6Sas4bNh4BeZtb8YFRseJOm14r5xjVVj LlcJEAnGOnQSAocVyhtZ5lZQBAc8jJHJ14n9ov8giDSJcBEmWXAAB8+K1ZHHBYksZWvPhH EH39VIPRAAx9Z1F+svSDumWEQmnaB4Ra4S3W6KDBms7TQO0K+7CNDR1r13QYeg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631792866; a=rsa-sha256; cv=none; b=EOpIutCD0+oEkh5NvRNin0jHhqkcUrAyy47e0pyTP8dJDJT79CYCkSabU9Kwlb5d8EEUju OhGbJgDZRj6vghzm6rdQVfmPR0s1nBjH7dri2RkLVX9HLHncI1co2CL1/KGTLCjlyruDKu 89QYgiq40TlfZHqn9H4pSmnTpIC/H6M86LW51hmX8yQsYYMgGu0gii+HashmQ3ZwhdONWn 5M8dbRORILWhAyCLLEtLeXvV/Bq9kJ5d3oH6b6p4Su9EBlBUoHqgfe7YMYHL7FZ3TLXyRa /42xKXOPfA981nwJmL28mFbHa2t7Qsc4G3hslYYas12R5+PvBEeRjxIz1f5YjQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=rammhold-de.20150623.gappssmtp.com header.s=20150623 header.b=Lzyml5sn; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 1.39 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=rammhold-de.20150623.gappssmtp.com header.s=20150623 header.b=Lzyml5sn; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: B70F22B294 X-Spam-Score: 1.39 X-Migadu-Scanner: scn0.migadu.com X-TUID: Fa4QxULiGmub I've recently became a little annoyed with a race condition in my current notmuch setup that originates from only having a single set of new tags for all notmuch-new(1) invocations. In the past I've mentioned this a couple of times in the IRC channel and now I got around to implement a basic version of this. I rougly process new messages like this: 1. new messages get the "new" tag through notmuch-new(1), 2. the post-new hook calls a series of scripts a) all mails with the "new" tag are processed by [muchclassy] which is responsible for applying tags to all mails from mailing lists following the schema list::org.kernel.vger.linux-kernel. b) all mails with the "new" tag and a tag that matches /^list::com\.github\./ are passed through another script of mine that queries the GitHub API and attaches tags (gh::closed, gh::merged, ...) to the thread. c) a notmuch-tag(1) --batch script is executed on all new mails that filters out some nosiy senders, groups mailing lists, filters out closed/merged GitHub PRs, ... and adds "unread"/"inbox"/... tags to mails that I want to see in my default inbox query. d) finally the "new" tag is removed from all mails. There are right now 8 mailboxes that I am retrieving mails from. I have a scheduled job that updates all my local Maildir's every couple of minutes. That one doesn't cause any issues on its own. But I also use IMAP IDLE to selectively update Maildir's as soon as a new mail arrives. If I receive mails on multiple Maildir's within a short period of time the above process is running into race-conditions since there is no way to distinguish mails that have just been marked new. All of them carry the same tags. In the worst case one of the last steps (2c/2d) would pick up the new mail before any of the actual classification has been executed. This "leaks" mails into my inbox which consequently can be overwhelming to look at. With this series I am able to give each notmuch-new(1) invocation a unique tag (think: new-$(uuidgen)). This doesn't (on its own) solve the entire story but is a first step in the "right direction" IMHO. I still have to wrap the entire workflow to propagate the unique tag to all the sub-commands (via. e.g an environment variable). I am posting this as an RFC to see what other users and the developers think about this approach and if anyone has solved a similar issue (in a different way). An alternative that I have considered is using a post-new hook that applies the unique tag and removes the default new tags. This would probably work but smells like a workaround / hack. There are two FIXME's in the docstrings of the new notmuch_config_values_from_string function as I didn't know what version this would possible first available in. Let me know what you think. Andi Andreas Rammhold (2): lib/config: introduce notmuch_config_values_from_string function CLI/notmuch: add --new-tags argument to notmuch-new(1) doc/man1/notmuch-new.rst | 7 +++++++ lib/config.cc | 12 +++++++++++- lib/notmuch.h | 14 ++++++++++++++ notmuch-new.c | 16 +++++++++++++++- test/T050-new.sh | 6 ++++++ test/T590-libconfig.sh | 22 ++++++++++++++++++++++ 6 files changed, 75 insertions(+), 2 deletions(-) -- 2.32.0