From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id IVS+LbqTm2BvBwEAgWs5BA (envelope-from ) for ; Wed, 12 May 2021 10:37:14 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id UHz7KLqTm2AoXAAA1q6Kng (envelope-from ) for ; Wed, 12 May 2021 08:37:14 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1BE352B449 for ; Wed, 12 May 2021 10:37:14 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id EC2FF2717B; Wed, 12 May 2021 04:37:08 -0400 (EDT) Received: from mailproxy05.manitu.net (mailproxy05.manitu.net [217.11.48.69]) by mail.notmuchmail.org (Postfix) with ESMTPS id 904E52717A for ; Wed, 12 May 2021 04:37:06 -0400 (EDT) Received: from localhost (200116b8605ca20012b0ea84d94bc93e.dip.versatel-1u1.de [IPv6:2001:16b8:605c:a200:12b0:ea84:d94b:c93e]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: michael@grubix.eu) by mailproxy05.manitu.net (Postfix) with ESMTPSA id 7CB4A1B616E1; Wed, 12 May 2021 10:37:05 +0200 (CEST) MIME-Version: 1.0 In-Reply-To: <20210512051013.bfrqx236n36bvqt6@feather> References: <60971dc77decb_22cb620817@natae.notmuch> <87bl9kjlgo.fsf@tethera.net> <8735uukd6c.fsf@tethera.net> <162065351282.5824.14225420932746058853.git@grubix.eu> <20210511062321.7rp3g64qfopaxedj@feather> <162071572578.3234.2579434453077254635.git@grubix.eu> <20210512051013.bfrqx236n36bvqt6@feather> Subject: Re: Is there a reason why the trashed flag is not synced? From: Michael J Gruber To: Reto Message-ID: <162080862357.6226.1101569739755943518.git@grubix.eu> Date: Wed, 12 May 2021 10:37:03 +0200 User-Agent: alot/0.9.1 Message-ID-Hash: CJZ4BN3V5UPZ4HATQ67DX4S3PABBF3RZ X-Message-ID-Hash: CJZ4BN3V5UPZ4HATQ67DX4S3PABBF3RZ X-MailFrom: michael@grubix.eu X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: notmuch@notmuchmail.org X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1620808634; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=0NEcqpLKekcmY9qDxwZWgPfIve9zGvW7mlr1n54mwa8=; b=V/hU4yw29VrFWR3I7T7uRxnd9oNVg+kGAtFSh8Mk8CuoJVqP24ZaFXmNLNGdeUQYsRqDCh 0l8SpS2ORmhx0cW7oTwHeI99GzoHPBv8KJMV5IJhc5PoxTQTGUBCgf6ry/x6JQX6NL7L5u +bIlbar/cigq5xW9Hkwlu/zF/pMXxa+sGlNj+XRLV+3rTe66jVCXQuwXOEV0SLQ5pqnDTE tKv57nN1SLuwCbzY+R343tsPK1xd5hv2Cc96QuiBR+hGkGC5kVp4zJz8yf53EIMQdo3vJ3 jTOO3akhxBT24jj13pWuvU3ikzFI805LywV3E0krIK8z6DfakPPwNiE28ZHcdw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1620808634; a=rsa-sha256; cv=none; b=QU1k6Cd8xh9PWTy+xOqbS3CDArIzu5DduGkUUsBKANgG7U2mi1K5VBUu42+ZnLjCrBbL67 UzI1QfZtdObHqUoHdhA9DtWGoJ5m0eh1NasitQpo2LmO7+iIs77kP5961Zu2CMOTRI3Ppk dpvjPW6Jg+qAq6CSt/Qk2fMQ5a0gyRzqQ7QXnO9J3POuAd0K4SEk/m2D/YrAv0ktTAn/H7 vamN1tpEdAign1B5ncg/lZhmjxr/ewXjfASlnSTps9lnDEIGcrFfpqjijsEOyg3W3WvkR5 plZpR143FT6sSiPyyUhWlYEwDLYjlFCQLjk4WBYEORuRPQ/gHgBQFfvFG3BkCw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -0.03 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 1BE352B449 X-Spam-Score: -0.03 X-Migadu-Scanner: scn0.migadu.com X-TUID: b7pjzMey/vVT Reto venit, vidit, dixit 2021-05-12 07:10:13: > On Tue, May 11, 2021 at 08:48:45AM +0200, Michael J Gruber wrote: > > If you sync gmail labels to exact copies in different folders then you're not > > holding it right, sorry ;) > > > Gmail is not an IMAP service; it has an IMAP API which exposes labels as > > folders, with all the caveats which this implies. That's why there are > > better ways to sync Gmail with a notmuch mail store (gmailieer). > > Better is always a relative term. I prefer a standardized protocol over some > homebrew stuff. ??? The question is whether you talk Gmail API to Gmail or IMAP to Gmail. What is homebrew here? > > I don't think notmuch should break its design principle (1 message id, 1 > > message) just to work around a problem caused by a wrong sync procedure: > > that "delete instruction" is a result (merely: artifact) of speaking IMAP to Gmail. > > It's not only gmail, really any IMAP server. > If you have a message in multiple folders at the same time (which is entirely > reasonable, think "todo" + where it actually should go after processing). > They all have the same message ID. > > Only because you delete the copy from todo doesn't mean that you want it gone > from other places as well. > > Sure, you could move instead of copying... > But that means that you may now need to hunt through multiple folders instead of > being able to grab the message from the actual mbox it should be in. > > IMAP doesn't impose a workflow, just because yours happens to be a certain way > doesn't make it the gold standard ;) Exactly, those arguments are turning into the ridiculous. The point is that folders are neither labels nor tags. You use an IMAP-sync tool to bolt on the tag concept onto a folder structure. This creates problems which only the sync tool can solve, not the indexer (notmuch), because it is the sync tool which translates the IMAP folder structure (and IMAP flags) into a maildir store (and maildir flags). Indeed, IMAP even has extended flags but support on the IMAP client side is typically suboptimal, so people don't use them at all or not to a great extent; they would be a good match for notmuch tags. Now, the only point at which notmuch is involved is indexing the maildir. This includes synchronizing the maildir flags to notmuch tags - and currently notmuch fails to synchronize one flag from the maildir standard. Felipe and I suggest to reinstate that. Note that since notmuch identifies messages by mid and one mid can correspond to multiple files (e.g. due to folder copies, list copies, sent/cc) it has to decide which file to index. Currently, notmuch assumes the content to be identical (it indexes the content of one file) but takes a union of reference headers and flags from all files. "Union of [boolean] flags" is the real issue here: AND or OR? Currently it's OR for all flags, and this happens to work for most workflows because the tag "unread" does not correspond to a maildir flag - rather, maildir has "S" (seen) so that OR of seen maildir flags decides whether notmuch does not tag the message "unread". Notice how this imposes a "gold standard workflow" on how you use read/unread on multiple copies of the same message? On the IMAP side, marking 1 copy of your message read or unread does not change the other one (much to the dismay of users who want to use automatic filters for sorting). Sync that (mbsync or such) two your maildir store and you one message file with "S", the other one without. Index with notmuch and *the* message will not be "unread" (even though one copy is not "Seen" per maildir flag). Toggle the "unread" tag with notmuch and all copies or none will have "S"; syncing back to the IMAP side will correspondingly mark all copies read or all unread. So, unless you force notmuch to sync tags back to maildir flags (e.g. by toggling them) it does not even mark all copies S even though the tag (absence of "unread") would warrant that. But whether this is the correct representation of "S" as "not unread" really depends on your workflow, that is your usage of copies und your usage of that flag. And yet, notmuch does sync it rather than bailing out because it requires a decision. In the very same way as marking one copy read marks all read (on the notmuch side as opposed to the IMAP side), why shouldn't the same be true for deleted? Because users with an IMAP mindset will complain about one but not the other, even though they are analogous... That's why I suggest the softer approach of treating the union of "T" as and AND: tag a message as "trashed" if all its copies have maildir flag "T". That way, users with multiple copies don't shoot themselves in the foot (and don't complain to us). If you tag a message as "trashed", all copies get "T", of course. This should give the best user experience: Once an IMAP server or IMAP sync tool expunges "T" messages notmuch will merely notices a missing file, unless it was the last one in which case it notices message deletion (and in which case it showed "trashed" before because it was the only one and had "T"). But really, if you use folders as a tag substitute you should think about moving files to a trash folder rather than marking them trashed, in which case we don't have to worry about "T". (Notmuch's lack of T-syncing made me do this and it works fine.) Michael