* Re: notmuchsync: handling of the deleted tag [not found] <87bp7vewa5.fsf@raven.defaultvalue.org> @ 2010-09-20 15:09 ` Sebastian Spaeth 2010-09-21 2:25 ` Rob Browning 0 siblings, 1 reply; 26+ messages in thread From: Sebastian Spaeth @ 2010-09-20 15:09 UTC (permalink / raw) To: Rob Browning; +Cc: Notmuch developer list On 2010-09-18, Rob Browning wrote: > I've been trying notmuchsync, and it's quite helpful, but I just ran > across something that caused me a bit of concern. I noticed that some > messages were marked deleted that I'm fairly certain I didn't intend to > delete. > > My suspicion is that this is because I received two copies of the > message and I marked one of them for deletion (expiry) in Gnus, but not > the other one. Since notmuch doesn't normally distinguish between two > messages with the same id, I suspected that notmuchsync might not notice > either. Is that possible? Yes, that is certainly possible. notmuchsync checks if a file should be deleted ("pruning") like this: 1)Run notmuchsync --revsync (or -r) and it will examine your mail files. If a mail is trashed (expired) according to maildir flags in its filename, it will add the tag "deleted" to the mail's id in the notmuch database. If you have multiple copies with the same message that tag is now associated with the mail id, not the actual file. 2) Run notmuchsync --prune which runs the query: "tag:delete or tag:deleted or tag:maildir::trashed" by default. To which notmuch returns a list of messages and a corresponding filename for each id. If there are multiple messages with the same id, I think it will return the first filename that was associated with that mail id, so it is kind of random which file location gets returned. The returned file names are associated with the "deleted" tag - and thus - unlinked. I am unsure how to handle this in a better way. What should notmuchsync --prune do if it finds multiple mail files that are associated with a "deleted" tag? And what should --revsync do when it finds a mail file that is marked as expired. As long as notmuch only offers one data entry per mail id, there will remain some ambiguity as to how to handle these. If notmuch gave me at least all filenames that are associated with a mail id, I could introduce a command line option "--prune --safe" which would Sebastian P.S. CC'ing the notmuch list as this is potentially an issue that affects people. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-20 15:09 ` notmuchsync: handling of the deleted tag Sebastian Spaeth @ 2010-09-21 2:25 ` Rob Browning 2010-09-21 9:44 ` Sebastian Spaeth 2010-11-12 1:27 ` notmuchsync: handling of the deleted tag Carl Worth 0 siblings, 2 replies; 26+ messages in thread From: Rob Browning @ 2010-09-21 2:25 UTC (permalink / raw) To: Sebastian Spaeth; +Cc: Notmuch developer list Sebastian Spaeth <Sebastian@SSpaeth.de> writes: > I am unsure how to handle this in a better way. What should notmuchsync > --prune do if it finds multiple mail files that are associated with a > "deleted" tag? Conceptually what I'd like for it to do, is reference count -- only mark the message deleted if every occurrence (across all maildirs) is marked trashed (T). Though even there I can imagine corner cases: imagine that notmuch doesn't initially see all your maildirs -- perhaps because you're using a folder filter in offlineimap, and so there are untrashed copies in the maildirs it hasn't seen yet. But reference counting of everything you can see is probably still better than what we have now, if it's feasible. > And what should --revsync do when it finds a mail file that is marked > as expired. Actually revsync is all I use so far, so that's what I'm talking about above. > If notmuch gave me at least all filenames that are associated with a > mail id, I could introduce a command line option "--prune --safe" which would > Sebastian Looks like you got cut off there. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-21 2:25 ` Rob Browning @ 2010-09-21 9:44 ` Sebastian Spaeth 2010-09-21 13:15 ` Mike Kelly 2010-09-22 0:30 ` Rob Browning 2010-11-12 1:27 ` notmuchsync: handling of the deleted tag Carl Worth 1 sibling, 2 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-09-21 9:44 UTC (permalink / raw) To: Rob Browning; +Cc: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 2432 bytes --] On 2010-09-21, Rob Browning wrote: > Conceptually what I'd like for it to do, is reference count -- only mark > the message deleted if every occurrence (across all maildirs) is marked > trashed (T). Right, but that is trickier than might appear at first sight. I parse those file names which notmuch is giving me (by default for mails from the last 30 days, as reparsing ALL your mails every time would be horribly expensive and often unneeded). notmuch is only able to give me one file name (+path) per mail id, so that is what I examine. If that is the one copy that has the mail dir flag "expired/trashed", I tag the message as "deleted". There is little else that notmuchsync can do here. I ask for new messages and their file path and if they are expired mark it as such. I have no way of finding out if there are other mails with the same mail id from notmuch (unless I am very much mistaken). Doing reference counting would require me/notmuchsync to parse ALL your mails by itself and finding out the often horribly mail id from the mail headers myself... something that notmuchsync does not want to get into. See the problem? I could do reference counting if notmuch were able to tell me how many file names/paths are associated with a mail id. > Though even there I can imagine corner cases: imagine that notmuch > doesn't initially see all your maildirs -- perhaps because you're using > a folder filter in offlineimap, and so there are untrashed copies in the > maildirs it hasn't seen yet. Right that would be a problem. But I cannot do reference counting unless notmuch can give me the number of copies it knows about for a given mail id (and internally it does know all associated file paths, so it would be a simple API extension a la, "get_all_message_file_paths" or similar, or a get_number_of_mail_files(mailid) to start with. > > And what should --revsync do when it finds a mail file that is marked > > as expired. What should notmuch do BTW if there are 2 copies and 1 is expired and 1 not? Mark as "deleted" or not? > Looks like you got cut off there. Right, it was 5pm and I left the computer :). I had intented to rant about the deficiencies of the notmuch 1 document per mail id approach here, but I don't see a better approach. All that would be useful from the notmuch side is to get all associated filenames with a mail id. sebastian [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-21 9:44 ` Sebastian Spaeth @ 2010-09-21 13:15 ` Mike Kelly 2010-09-21 13:30 ` Jameson Rollins 2010-09-22 0:30 ` Rob Browning 1 sibling, 1 reply; 26+ messages in thread From: Mike Kelly @ 2010-09-21 13:15 UTC (permalink / raw) To: notmuch On Tue, 21 Sep 2010 11:44:29 +0200 Sebastian Spaeth <Sebastian@SSpaeth.de> wrote: > Right that would be a problem. But I cannot do reference counting > unless notmuch can give me the number of copies it knows about for a > given mail id (and internally it does know all associated file paths, > so it would be a simple API extension a la, > "get_all_message_file_paths" or similar, or a > get_number_of_mail_files(mailid) to start with. I agree, and such an API would be useful for other things too. For example, I could probably implement a rudamentary gmail syncer with that. I may just try to make a patch to do this soon if nobody beats me to it. -- Mike Kelly ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-21 13:15 ` Mike Kelly @ 2010-09-21 13:30 ` Jameson Rollins 2010-09-22 8:53 ` Sebastian Spaeth 0 siblings, 1 reply; 26+ messages in thread From: Jameson Rollins @ 2010-09-21 13:30 UTC (permalink / raw) To: Mike Kelly, notmuch [-- Attachment #1: Type: text/plain, Size: 420 bytes --] On Tue, 21 Sep 2010 09:15:44 -0400, Mike Kelly <pioto@pioto.org> wrote: > I agree, and such an API would be useful for other things too. For > example, I could probably implement a rudamentary gmail syncer with > that. I may just try to make a patch to do this soon if nobody beats me > to it. I think this is related to the requests that I've made to have access to all versions of duplicate messages as well. jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-21 13:30 ` Jameson Rollins @ 2010-09-22 8:53 ` Sebastian Spaeth 0 siblings, 0 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-09-22 8:53 UTC (permalink / raw) To: notmuch [-- Attachment #1: Type: text/plain, Size: 267 bytes --] On 2010-09-21, Jameson Rollins wrote: > I think this is related to the requests that I've made to have access to > all versions of duplicate messages as well. Absolutely yes. This would allow solving your problem with duplicate messages as well, I think. Sebastian [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-21 9:44 ` Sebastian Spaeth 2010-09-21 13:15 ` Mike Kelly @ 2010-09-22 0:30 ` Rob Browning 2010-09-23 7:30 ` Sebastian Spaeth ` (2 more replies) 1 sibling, 3 replies; 26+ messages in thread From: Rob Browning @ 2010-09-22 0:30 UTC (permalink / raw) To: Sebastian Spaeth; +Cc: Notmuch developer list Sebastian Spaeth <Sebastian@SSpaeth.de> writes: > On 2010-09-21, Rob Browning wrote: >> Conceptually what I'd like for it to do, is reference count -- only mark >> the message deleted if every occurrence (across all maildirs) is marked >> trashed (T). > > Right, but that is trickier than might appear at first sight. In general, I think that until/unless notmuchsync can be more assured of doing the right thing, and in particular, if the deleted tag is likely to become official, notmuchsync should default to not setting it. ...or at least, I'd prefer that. Then I can add --tag-deleted if/when I want to. Of course defaulting to --tag-deleted would also be OK, as long as there's a --no-tag-deleted. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-22 0:30 ` Rob Browning @ 2010-09-23 7:30 ` Sebastian Spaeth 2010-09-23 7:32 ` Sebastian Spaeth 2010-09-23 8:01 ` notmuchsync default behavior change (was: notmuchsync: handling of the deleted tag) Sebastian Spaeth 2 siblings, 0 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-09-23 7:30 UTC (permalink / raw) To: Rob Browning; +Cc: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1602 bytes --] On 2010-09-22, Rob Browning wrote: > In general, I think that until/unless notmuchsync can be more assured of > doing the right thing, and in particular, if the deleted tag is likely > to become official, notmuchsync should default to not setting it. > > ...or at least, I'd prefer that. Then I can add --tag-deleted if/when I > want to. Of course defaulting to --tag-deleted would also be OK, as > long as there's a --no-tag-deleted. In my workflow it does the right thing as I only "expire" a message if I am happy that it actually will be deleted. The situation is a bit less scary that it seemed to me: if a notmuch --revsync examines a mail file, it is the exact copy that notmuch will point to, and it will mark that as "deleted" (and eventually "--prune" can delete that very copy). A new run by notmuchsync --revsync will then be told to look at the second copy if the first is gone and it will discover that it does not have the maildir flag "expired" and thus remove the "deleted" tag in the notmuch db. As such, the tag should reflect the actual status of the mail file that notmuch points to. But I can imagine circumstances in which there would be an erroneous tag be set and I don't want notmuchsync to accidently delete anyones emails, so I'll disable the "deleted" tag synchronization by default and only make it an option (that I will use). Solving this in a proper way will require support from the notmuch API though. Sebastian P.S. cworth made clear that he preferred if there were no "official" tags or some tags that are more queal than others. [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-22 0:30 ` Rob Browning 2010-09-23 7:30 ` Sebastian Spaeth @ 2010-09-23 7:32 ` Sebastian Spaeth 2010-09-23 8:01 ` notmuchsync default behavior change (was: notmuchsync: handling of the deleted tag) Sebastian Spaeth 2 siblings, 0 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-09-23 7:32 UTC (permalink / raw) To: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 421 bytes --] On 2010-09-22, Rob Browning wrote: > In general, I think that until/unless notmuchsync can be more assured of > doing the right thing, and in particular, if the deleted tag is likely > to become official, notmuchsync should default to not setting it. BTW, I consider --revsync as a kludge anyway that belongs into notmuch proper. There are patches to achieve that kind of synchronization on the mailing list. Sebastian [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* notmuchsync default behavior change (was: notmuchsync: handling of the deleted tag) 2010-09-22 0:30 ` Rob Browning 2010-09-23 7:30 ` Sebastian Spaeth 2010-09-23 7:32 ` Sebastian Spaeth @ 2010-09-23 8:01 ` Sebastian Spaeth 2 siblings, 0 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-09-23 8:01 UTC (permalink / raw) To: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1838 bytes --] On 2010-09-22, Rob Browning wrote: > In general, I think that until/unless notmuchsync can be more assured of > doing the right thing, and in particular, if the deleted tag is likely > to become official, notmuchsync should default to not setting it. > ...or at least, I'd prefer that. Then I can add --tag-deleted if/when I > want to. Of course defaulting to --tag-deleted would also be OK, as > long as there's a --no-tag-deleted. notmuchsync does not want to delete anyones' e-mails by accident and although I believe that syncing the "deleted" tag should usually be safe, I can think of situations with multiple copies of a mail where things could go wrong. I therefore disabled syncing of the maildir flag "trashed" and the notmuch tag "deleted" by default. It has to be explicitly be enabled with the new --sync-deleted option. Sebastian ------------------------------------------------------------------ For reference, this could be a szenario where things turn out bad: - Send a mail with FCC to your Sent folder and BCC yourself. You have then 2 mail copies with the same mail id (1 notmuch entry) - "Expire" your BCC copy via some mail/webclient, it gets maildir flag "T" - notmuch --revsync will examine new mails and in case the BCC copy is what notmuch points to, notmuchsync will then add the notmuch tag "deleted" to that mail id. - notmuchsync --prune deletes the BCC mail copy but does not remove the "deleted" tag in the notmuch db (it probably should for safety reasons). - 'notmuch new' will now recognize the BCC copy as removed and point to the "Sent/FCC" copy as the current file, but it still carries the "deleted" tag for the mail. - notmuchsync --prune will happily delete your Sent/FCC copy of the mail, or notmuchsync --sync will mark your Sent/FCC copy with the "expired" maildir flag. [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-09-21 2:25 ` Rob Browning 2010-09-21 9:44 ` Sebastian Spaeth @ 2010-11-12 1:27 ` Carl Worth 2010-11-12 7:30 ` Sebastian Spaeth 1 sibling, 1 reply; 26+ messages in thread From: Carl Worth @ 2010-11-12 1:27 UTC (permalink / raw) To: Rob Browning, Sebastian Spaeth; +Cc: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1688 bytes --] On Mon, 20 Sep 2010 21:25:13 -0500, Rob Browning <rlb@defaultvalue.org> wrote: > Sebastian Spaeth <Sebastian@SSpaeth.de> writes: > Conceptually what I'd like for it to do, is reference count -- only mark > the message deleted if every occurrence (across all maildirs) is marked > trashed (T). [I'm going back through this old notmuchsync thread after having worked to merge and improve some related synchronization featured.] The above idea sounds quite interesting and would be possible in the current approach. The only thing I don't like is that we would have some very detailed knowledge about the semantics of "deleted" hard-coded deep in the library. I already don't like the fact that the library has a hard-coded list with a handful of mappings between flags and tags. So, what we probably need here is for the user to be able to configure the mapping and in a fairly sophisticated way: 'R' on _any_ filename -> "replied" tag gets added 'T' on _all_ filenames -> "deleted" tag gets added 'S' on _any_ filename -> "unread" tag gets removed That would be expressible with just a couple of pieces of punctuation: R=replied;T*=deleted;S=~unread So maybe something like that? > > If notmuch gave me at least all filenames that are associated with a > > mail id, I could introduce a command line option "--prune --safe" > > which would This feature is at least implemented in libnotmuch at least. Hopefully things just keep getting better and better. I am looking forward to some feedback from users of notmuchsync who try out the new maildir.synchronize_flags support in notmuch itself. -Carl -- carl.d.worth@intel.com [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-12 1:27 ` notmuchsync: handling of the deleted tag Carl Worth @ 2010-11-12 7:30 ` Sebastian Spaeth 2010-11-12 21:04 ` Dirk Hohndel 2010-11-12 21:16 ` Carl Worth 0 siblings, 2 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-11-12 7:30 UTC (permalink / raw) To: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1033 bytes --] On Thu, 11 Nov 2010 17:27:34 -0800, Carl Worth <cworth@cworth.org> wrote: > So, what we probably need here is for the user to be able to configure > the mapping and in a fairly sophisticated way: > > 'R' on _any_ filename -> "replied" tag gets added > 'T' on _all_ filenames -> "deleted" tag gets added > 'S' on _any_ filename -> "unread" tag gets removed > So maybe something like that? Maybe, but that sounds like a horribly complex configuration, in which the user has to really think through what he wants (and can still make blunders). :) > > > If notmuch gave me at least all filenames that are associated with a > > > mail id, I could introduce a command line option "--prune --safe" > > > which would Right, you pushed the ball in my court. The only problem is that -- with the arrival of maildir sync -- I lost my motivation to work on notmuchsync. Seriously, what does notmuchsync still provide that notmuch cannot do? I wonder if I shouldn't stick a "deprecated" warning on it. Sebastian [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-12 7:30 ` Sebastian Spaeth @ 2010-11-12 21:04 ` Dirk Hohndel 2010-11-13 0:52 ` Carl Worth 2010-11-12 21:16 ` Carl Worth 1 sibling, 1 reply; 26+ messages in thread From: Dirk Hohndel @ 2010-11-12 21:04 UTC (permalink / raw) To: Sebastian Spaeth, Notmuch developer list On Fri, 12 Nov 2010 08:30:36 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote: > On Thu, 11 Nov 2010 17:27:34 -0800, Carl Worth <cworth@cworth.org> wrote: > > So, what we probably need here is for the user to be able to configure > > the mapping and in a fairly sophisticated way: > > > > 'R' on _any_ filename -> "replied" tag gets added > > 'T' on _all_ filenames -> "deleted" tag gets added > > 'S' on _any_ filename -> "unread" tag gets removed > > So maybe something like that? > > Maybe, but that sounds like a horribly complex configuration, in which > the user has to really think through what he wants (and can still make > blunders). :) > > > > > If notmuch gave me at least all filenames that are associated with a > > > > mail id, I could introduce a command line option "--prune --safe" > > > > which would > > Right, you pushed the ball in my court. The only problem is that -- with > the arrival of maildir sync -- I lost my motivation to work on > notmuchsync. Seriously, what does notmuchsync still provide that notmuch > cannot do? I wonder if I shouldn't stick a "deprecated" warning on it. Please don't! I use it all the time: I use it to archive mail that I don't want on an imap server any more, but in a local maildir. Can't do that with notmuch. I use it to prune mail that has the deleted tag. Again not something that notmuch supports. So please don't stop supporting your very important (to me) tool! /D ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-12 21:04 ` Dirk Hohndel @ 2010-11-13 0:52 ` Carl Worth 2010-11-14 22:21 ` Sebastian Spaeth 2010-11-15 3:01 ` Jameson Rollins 0 siblings, 2 replies; 26+ messages in thread From: Carl Worth @ 2010-11-13 0:52 UTC (permalink / raw) To: Dirk Hohndel, Sebastian Spaeth, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 3225 bytes --] On Fri, 12 Nov 2010 13:04:16 -0800, Dirk Hohndel <hohndel@infradead.org> wrote: > > notmuchsync. Seriously, what does notmuchsync still provide that notmuch > > cannot do? I wonder if I shouldn't stick a "deprecated" warning on it. > > Please don't! I use it all the time: Excellent. I love getting more feedback from users so that we can understand things better. > I use it to archive mail that I don't want on an imap server any more, > but in a local maildir. Can't do that with notmuch. Can you describe this use case in a bit more detail? I'd like to understand what needs to be done to support this. > I use it to prune mail that has the deleted tag. Again not something > that notmuch supports. This one is a simple one-liner as of notmuch 0.4: notmuch search --output=files tag:deleted -print0 | xargs -0 rm And that's something we could even add to a keybinding in the emacs interface[*]. Meanwhile, I know people have asked for keybindings to even just add the "deleted" tag before. What I still want for that is good support for being able to configure tags that cause mails to be automatically omitted from search results---unless the user is explicitly searching for mails with that tag. So for example, I would add "deleted" and "spam" to such a list and never see mails with that tag unless I go looking for a mis-tagged email with a search like "from:someone and tag:spam". I think the above should be fairly easy. One way to do it is to just implicitly add "and not (tag:deleted or tag:spam)" to all searches. And then simply omit the relevant portion of this if the original search terms happen to mention tag:deleted or tag:spam. This is what sup does, for example. -Carl [*] Personally, I don't want a keybinding for actual deletion of mail files. As long as nothing appears in searches, then I'd prefer the tagged messages not be deleted right away. Instead, I'd like a cron job to delete appropriately tagged messages after some safety interval (a month, say). For that, I'd like better support for specifying message dates of course, (that's been on my todo list for some time). But really, I care about the date of the tag, not the date of the message. We don't save information like that, but the cron script could do something cheesy like implement counters in tags: # XXX: Entirely untested! # Days after a mail is tag:deleted when the mail will be deleted EXPIRATION=30 for days in $(seq 0 $((EXPIRATION - 1))); do notmuch tag +expire-$((days)) -expire-$((days + 1)) tag:expire-$(days + 1)) done notmuch search --output=files tag:expire-0 and tag:deleted -print0 | xargs -0 rm notmuch tag -expire-0 tag:expire-0 notmuch tag +expire-$EXPIRATION +expiring tag:deleted and not tag:expiring notmuch tag -expiring not tag:deleted OK. Looking at that---that's *really* cheesy. Perhaps we should come up with a sane way to do this. Hooks on the addition or deletion of the "deleted" tag that set/cleared a date value in the message document in the database might be much cleaner. But the above might actually work for now. Anyone brave enough to test it? -- carl.d.worth@intel.com [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-13 0:52 ` Carl Worth @ 2010-11-14 22:21 ` Sebastian Spaeth 2010-11-14 22:36 ` Jameson Rollins 2010-11-15 1:23 ` Jeff Richards 2010-11-15 3:01 ` Jameson Rollins 1 sibling, 2 replies; 26+ messages in thread From: Sebastian Spaeth @ 2010-11-14 22:21 UTC (permalink / raw) To: Carl Worth, Dirk Hohndel, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 426 bytes --] On Fri, 12 Nov 2010 16:52:51 -0800, Carl Worth wrote: > On Fri, 12 Nov 2010 13:04:16 -0800, Dirk Hohndel <hohndel@infradead.org> wrote: > > Please don't! I use it all the time: Thanks :). > This one is a simple one-liner as of notmuch 0.4: > > notmuch search --output=files tag:deleted -print0 | xargs -0 rm Uhh, that is a neat line, and a neat trick. (is it on the emacstips page already? ;-)) Sebastian [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-14 22:21 ` Sebastian Spaeth @ 2010-11-14 22:36 ` Jameson Rollins 2010-11-15 1:23 ` Jeff Richards 1 sibling, 0 replies; 26+ messages in thread From: Jameson Rollins @ 2010-11-14 22:36 UTC (permalink / raw) To: Sebastian Spaeth, Carl Worth, Dirk Hohndel, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 417 bytes --] On Sun, 14 Nov 2010 23:21:41 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote: > > This one is a simple one-liner as of notmuch 0.4: > > > > notmuch search --output=files tag:deleted -print0 | xargs -0 rm > > Uhh, that is a neat line, and a neat trick. (is it on the emacstips page > already? ;-)) It's not an emacs-specific tip! I guess the web page needs a "general tips" section... jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-14 22:21 ` Sebastian Spaeth 2010-11-14 22:36 ` Jameson Rollins @ 2010-11-15 1:23 ` Jeff Richards 2010-11-15 3:03 ` Jameson Rollins 1 sibling, 1 reply; 26+ messages in thread From: Jeff Richards @ 2010-11-15 1:23 UTC (permalink / raw) To: Notmuch developer list On Sun, 14 Nov 2010 23:21:41 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote: > This one is a simple one-liner as of notmuch 0.4: > > > > notmuch search --output=files tag:deleted -print0 | xargs -0 rm > Having just switched from notmuchsync the synchronizing power of 0.5 and being as how I'm missing the pruning feature I gave this a try and found that I ended up with rm complaining that the file name length being too long or simply file not found. Seems like the xargs -0 rm portion was choking as all the file names were being contcatenated. Perhaps the null character line ending is missing? At this point I haven't investigated further. So for anyone else who is stuck adjusting the one liner like notmuch search --output=files tag:deleted -print0 | xargs -d '\n' rm worked for me. PS Thanks to Sebastian and Carl and everyone else who has built notmuch I'm really enjoying using it! :) Jeff > Uhh, that is a neat line, and a neat trick. (is it on the emacstips page > already? ;-)) > > Sebastian Non-text part: application/pgp-signature > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch -- Jeff Richards jrichards@revenuewire.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-15 1:23 ` Jeff Richards @ 2010-11-15 3:03 ` Jameson Rollins 2010-11-15 15:24 ` servilio 0 siblings, 1 reply; 26+ messages in thread From: Jameson Rollins @ 2010-11-15 3:03 UTC (permalink / raw) To: Jeff Richards, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1089 bytes --] On Sun, 14 Nov 2010 17:23:57 -0800, Jeff Richards <jrichards@revenuewire.com> wrote: > Having just switched from notmuchsync the synchronizing power of 0.5 and > being as how I'm missing the pruning feature I gave this a try and found > that I ended up with rm complaining that the file name length being too > long or simply file not found. Seems like the > xargs -0 rm > portion was choking as all the file names were being contcatenated. > Perhaps the null character line ending is missing? At this point I > haven't investigated further. I think the 'print0' might have been a mistake on Carl's part. I don't think that option is actually supported by notmuch. In which case xargs -0 isn't going to work as expected because there are no null characters in the input stream to use as delimiters. > So for anyone else who is stuck adjusting the one liner like > notmuch search --output=files tag:deleted -print0 | xargs -d '\n' rm This is working because the input stream is newline delimited. So I think the -print0 is confusing the issue. jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-15 3:03 ` Jameson Rollins @ 2010-11-15 15:24 ` servilio 2010-11-15 16:48 ` Daniel Kahn Gillmor 0 siblings, 1 reply; 26+ messages in thread From: servilio @ 2010-11-15 15:24 UTC (permalink / raw) To: Jameson Rollins; +Cc: Notmuch developer list On 14 November 2010 22:03, Jameson Rollins <jrollins@finestructure.net> wrote: [...] > I think the 'print0' might have been a mistake on Carl's part. I don't > think that option is actually supported by notmuch. In which case xargs > -0 isn't going to work as expected because there are no null characters > in the input stream to use as delimiters. > >> So for anyone else who is stuck adjusting the one liner like >> notmuch search --output=files tag:deleted -print0 | xargs -d '\n' rm > > This is working because the input stream is newline delimited. So I > think the -print0 is confusing the issue. And "xargs -L1 ..." will solve the issue in a nicer way. So the above would be: notmuch search --output=files tag:deleted | xargs -L1 rm Servilio ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-15 15:24 ` servilio @ 2010-11-15 16:48 ` Daniel Kahn Gillmor 0 siblings, 0 replies; 26+ messages in thread From: Daniel Kahn Gillmor @ 2010-11-15 16:48 UTC (permalink / raw) To: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 436 bytes --] On 11/15/2010 10:24 AM, servilio wrote: > And "xargs -L1 ..." will solve the issue in a nicer way. So the above would be: > > notmuch search --output=files tag:deleted | xargs -L1 rm Assuming none of your mail filenames have newlines or trailing whitespace, that is. I'd like to make that kind of assumption, myself. But I've certainly seen files that violate it (not from any maildir implementation, though). --dkg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 900 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-13 0:52 ` Carl Worth 2010-11-14 22:21 ` Sebastian Spaeth @ 2010-11-15 3:01 ` Jameson Rollins 2010-11-15 19:58 ` Carl Worth 1 sibling, 1 reply; 26+ messages in thread From: Jameson Rollins @ 2010-11-15 3:01 UTC (permalink / raw) To: Carl Worth, Dirk Hohndel, Sebastian Spaeth, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 482 bytes --] On Fri, 12 Nov 2010 16:52:51 -0800, Carl Worth <cworth@cworth.org> wrote: > This one is a simple one-liner as of notmuch 0.4: > > notmuch search --output=files tag:deleted -print0 | xargs -0 rm Is -print0 actually a command that notmuch accepts? It's not documented as far as I can tell. And it doesn't seem to affect the output of the command, i.e. whether or not it's used the output is still newline delimited. I'm not seeing it in the source either. jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-15 3:01 ` Jameson Rollins @ 2010-11-15 19:58 ` Carl Worth 2010-11-15 20:16 ` Daniel Kahn Gillmor 0 siblings, 1 reply; 26+ messages in thread From: Carl Worth @ 2010-11-15 19:58 UTC (permalink / raw) To: Jameson Rollins, Dirk Hohndel, Sebastian Spaeth, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1265 bytes --] On Sun, 14 Nov 2010 22:01:09 -0500, Jameson Rollins <jrollins@finestructure.net> wrote: > On Fri, 12 Nov 2010 16:52:51 -0800, Carl Worth <cworth@cworth.org> wrote: > > This one is a simple one-liner as of notmuch 0.4: > > > > notmuch search --output=files tag:deleted -print0 | xargs -0 rm > > Is -print0 actually a command that notmuch accepts? It's not documented > as far as I can tell. And it doesn't seem to affect the output of the > command, i.e. whether or not it's used the output is still newline > delimited. I'm not seeing it in the source either. No, it's not something implemented. I'm surprised I spouted such nonsense above. I think what happened is that I first typed a command line which would actually work, (for filenames without embedded newline characters): notmuch search --output=files tag:deleted | xargs rm And then a habit kicked in from typing many find/xargs commands and I retrofitted the command with -print0 and -0 as I originally sent it out. Should we perhaps add a -0 option so that one could actually reliably handle filenames with newlines in a case like this? Or can we just assert that such filenames are insane and *should* cause problems? -Carl -- carl.d.worth@intel.com [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-15 19:58 ` Carl Worth @ 2010-11-15 20:16 ` Daniel Kahn Gillmor 0 siblings, 0 replies; 26+ messages in thread From: Daniel Kahn Gillmor @ 2010-11-15 20:16 UTC (permalink / raw) To: Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 723 bytes --] On 11/15/2010 02:58 PM, Carl Worth wrote: > Should we perhaps add a -0 option so that one could actually reliably > handle filenames with newlines in a case like this? Or can we just > assert that such filenames are insane and *should* cause problems? it's not just newlines. Without using the -L1 flag to xargs (suggested earlier on this list), it's actually any filename with whitespace. with -L1, newlines and trailing whitespace are still problematic. newlines and trailing whitespace are unarguably insane in file names. But they happen in the wild, and it would be a shame if notmuch broke with them. i think offering -print0 or -0 to provide null byte delimiters is a good proposal. --dkg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 900 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-12 7:30 ` Sebastian Spaeth 2010-11-12 21:04 ` Dirk Hohndel @ 2010-11-12 21:16 ` Carl Worth 2010-11-14 22:30 ` Sebastian Spaeth 1 sibling, 1 reply; 26+ messages in thread From: Carl Worth @ 2010-11-12 21:16 UTC (permalink / raw) To: Sebastian Spaeth, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 2085 bytes --] On Fri, 12 Nov 2010 08:30:36 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote: > On Thu, 11 Nov 2010 17:27:34 -0800, Carl Worth <cworth@cworth.org> wrote: > > 'R' on _any_ filename -> "replied" tag gets added > > 'T' on _all_ filenames -> "deleted" tag gets added > > 'S' on _any_ filename -> "unread" tag gets removed > > So maybe something like that? > > Maybe, but that sounds like a horribly complex configuration, in which > the user has to really think through what he wants (and can still make > blunders). :) Yes. But a common user configurations could be as simple as: R:replied; t:todo; d:done; or so. And that would be hugely powerful because it would allow for remote synchronization of tags with tools that exist *today* (and perhaps[*] even through imap servers where the user cannot install custom software). [*] This is a point I'm not clear on. Would a tool like offlineimap be able to push custom filenames through to the remote end with standard imap servers? But we could also have a well-documented default configuration that happens to include two of the more complex cases: R:replied; T* : deleted; S : ~unread; So questions of syntax aside, I'm not convinced that this idea is insane. And if it covers all of the synchronization needs that people really have. > Right, you pushed the ball in my court. The only problem is that -- with > the arrival of maildir sync -- I lost my motivation to work on > notmuchsync. Hehe. Oh well, someone else will have to go play with notmuch_message_get_filenames. I think the people interested in writing a "gmail importer" want this as well. > Seriously, what does notmuchsync still provide that notmuch > cannot do? I wonder if I shouldn't stick a "deprecated" warning on it. That's actually a question I wanted to ask you. If notmuch 0.5 actually does everything that notmuchsync did, then that's great news. If there is any missing functionality, then let's get that into notmuch. Thanks for the discussion. -Carl -- carl.d.worth@intel.com [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-12 21:16 ` Carl Worth @ 2010-11-14 22:30 ` Sebastian Spaeth 2010-11-15 20:01 ` Carl Worth 0 siblings, 1 reply; 26+ messages in thread From: Sebastian Spaeth @ 2010-11-14 22:30 UTC (permalink / raw) To: Carl Worth, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 1556 bytes --] On Fri, 12 Nov 2010 13:16:53 -0800, Carl Worth wrote: > [*] This is a point I'm not clear on. Would a tool like offlineimap be > able to push custom filenames through to the remote end with standard > imap servers? Not sure. If it worked this would be really nice. But as most IMAP server support arbitrary flags to be added to a message, I would still think the best is to code a tool that talks IMAP and synchronizes notmuch tags with the IMAP server. I've never done it before and python's IMAP support is pretty crappy or I would alredy have given it a go :). > So questions of syntax aside, I'm not convinced that this idea is > insane. And if it covers all of the synchronization needs that people > really have. Yes, it would cover a lot of my needs too. Altough I am still aiming at propagating my notmuch flags via the IMAP server and back :). > That's actually a question I wanted to ask you. If notmuch 0.5 actually > does everything that notmuchsync did, then that's great news. If there > is any missing functionality, then let's get that into notmuch. Pruning used to be done nicely with notmuchsync, but you have given a nice example how that could be done with notmuch now. The one thing I can see now that notmuchsync still does is moving maessages to a folder when they match a tag. But that would be an equally short shell script that notmuch can do now. So, while I might still fix bugs, I do not think that notmuchsync will evolve a lot further (until I make it speak IMAP which I don't see coming in the near future) Sebastian [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: notmuchsync: handling of the deleted tag 2010-11-14 22:30 ` Sebastian Spaeth @ 2010-11-15 20:01 ` Carl Worth 0 siblings, 0 replies; 26+ messages in thread From: Carl Worth @ 2010-11-15 20:01 UTC (permalink / raw) To: Sebastian Spaeth, Notmuch developer list [-- Attachment #1: Type: text/plain, Size: 944 bytes --] On Sun, 14 Nov 2010 23:30:13 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote: > Not sure. If it worked this would be really nice. But as most IMAP > server support arbitrary flags to be added to a message, I would still > think the best is to code a tool that talks IMAP and synchronizes > notmuch tags with the IMAP server. I've never done it before and > python's IMAP support is pretty crappy or I would alredy have given it a > go :). Right. A tool that did synchronize notmuch tags by talking to an IMAP server would be something of interest to many people. But, from all the rumors I've heard, writing code to implement IMAP isn't much fun. So we'll have to wait for someone brave, I guess. In the meantime, I'll be happy enough to synchronize my mail with rsync, so I may very well implement the support for custom mapping of maildir-style filename flags to notmuch tags. -Carl -- carl.d.worth@intel.com [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2010-11-15 20:16 UTC | newest] Thread overview: 26+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <87bp7vewa5.fsf@raven.defaultvalue.org> 2010-09-20 15:09 ` notmuchsync: handling of the deleted tag Sebastian Spaeth 2010-09-21 2:25 ` Rob Browning 2010-09-21 9:44 ` Sebastian Spaeth 2010-09-21 13:15 ` Mike Kelly 2010-09-21 13:30 ` Jameson Rollins 2010-09-22 8:53 ` Sebastian Spaeth 2010-09-22 0:30 ` Rob Browning 2010-09-23 7:30 ` Sebastian Spaeth 2010-09-23 7:32 ` Sebastian Spaeth 2010-09-23 8:01 ` notmuchsync default behavior change (was: notmuchsync: handling of the deleted tag) Sebastian Spaeth 2010-11-12 1:27 ` notmuchsync: handling of the deleted tag Carl Worth 2010-11-12 7:30 ` Sebastian Spaeth 2010-11-12 21:04 ` Dirk Hohndel 2010-11-13 0:52 ` Carl Worth 2010-11-14 22:21 ` Sebastian Spaeth 2010-11-14 22:36 ` Jameson Rollins 2010-11-15 1:23 ` Jeff Richards 2010-11-15 3:03 ` Jameson Rollins 2010-11-15 15:24 ` servilio 2010-11-15 16:48 ` Daniel Kahn Gillmor 2010-11-15 3:01 ` Jameson Rollins 2010-11-15 19:58 ` Carl Worth 2010-11-15 20:16 ` Daniel Kahn Gillmor 2010-11-12 21:16 ` Carl Worth 2010-11-14 22:30 ` Sebastian Spaeth 2010-11-15 20:01 ` Carl Worth
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).