Message deletion wisdom

Message deletion wisdom

[Jacek Generowicz]

Hello,

Looking through the archives of this list, I see that the topic of
message deletion is, if not controversial, then at least non-trivial.

+ Requests for the provision of message deletion mechanisms have
  appeared many times, and that various patches addressing the issue
  have been submitted, and (IIUC) some of these are awaiting review
  and might possibly make it into an official release in the future.

+ Having the 'deleted' tag cause messages to be deleted is,
  apparently, fraught with peril of losing mail.


I have two questions:


1) Could you give me a summary of the dangers of allowing notmuch to
   actually delete mail?

2) How do you deal with mail that you would like to disappear from
   your maildirs/IMAP servers? (An Emacs-based slant would be
   particularle welcome.)


Thank you for your input.

Re: Message deletion wisdom

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]

On Tue, Apr 03 2012, Jacek Generowicz <jacek.generowicz@cern.ch> wrote:
> Looking through the archives of this list, I see that the topic of
> message deletion is, if not controversial, then at least non-trivial.
>
> + Requests for the provision of message deletion mechanisms have
>   appeared many times, and that various patches addressing the issue
>   have been submitted, and (IIUC) some of these are awaiting review
>   and might possibly make it into an official release in the future.
>
> + Having the 'deleted' tag cause messages to be deleted is,
>   apparently, fraught with peril of losing mail.

Hi, Jacek.  You are right on both of these points.  It has indeed been a
controversial topic.  However, I would say that we have reached a stasis
in terms of our response to this issue.  I'll try to summarize here.

I can say with a very high degree of certainly that notmuch will NEVER
gain the capability to actually delete mail files from disk itself.  It
is too risky for notmuch to be involved in that, and it's too easy to do
it outside of notmuch (e.g. "notmuch search --output=files tag:deleted |
xargs rm").  This has never really been up for discussion.

What *has* generated a lot of discussion is whether or not the emacs
interface should by default support key bindings that add a "deleted"
tag to messages, and how that tag should be handled.  I think we have
finally settled on notmuch NOT supporting such a key binding by default,
since it's easy enough for users to add the binding themselves if they
wish.

What we have added (or more accurately "are adding") though (thanks to
the work of Austin Clements and Mark Walters) is the ability to exclude
messages with a certain tag from search results.  This is like an
effective deletion, since messages can be completely omitted from search
results without having them actually be deleted from disk.  This is
obviously much safer, and a more generally useful functionality.  This
functionality is currently in beta, but will be full-featured in the
next release.

So in summary, if you would like to "delete" messages, you can:

 * add a key binding to emacs (or your favorite ui) to add a "deleted"
   tag to messages that you want to delete:

    (define-key notmuch-show-mode-map "d"
      (lambda ()
        (interactive)
        (notmuch-show-tag-message "+deleted")))

 * add exclusion for messages with the "deleted" tag:

    notmuch config set search.exclude_tags deleted

 * and if you really want them purged from disk, delete them manually
   with:

    notmuch search --output=files tag:deleted | xargs -l rm

I'll add a page about this to the wiki, since it comes up enough.

hth.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Message deletion wisdom

[Jani Nikula]

On Tue, 03 Apr 2012 16:32:04 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Tue, Apr 03 2012, Jacek Generowicz <jacek.generowicz@cern.ch> wrote:
> > Looking through the archives of this list, I see that the topic of
> > message deletion is, if not controversial, then at least non-trivial.
> >
> > + Requests for the provision of message deletion mechanisms have
> >   appeared many times, and that various patches addressing the issue
> >   have been submitted, and (IIUC) some of these are awaiting review
> >   and might possibly make it into an official release in the future.
> >
> > + Having the 'deleted' tag cause messages to be deleted is,
> >   apparently, fraught with peril of losing mail.
> 
> Hi, Jacek.  You are right on both of these points.  It has indeed been a
> controversial topic.  However, I would say that we have reached a stasis
> in terms of our response to this issue.  I'll try to summarize here.
> 
> I can say with a very high degree of certainly that notmuch will NEVER
> gain the capability to actually delete mail files from disk itself.  It
> is too risky for notmuch to be involved in that, and it's too easy to do
> it outside of notmuch (e.g. "notmuch search --output=files tag:deleted |
> xargs rm").  This has never really been up for discussion.

To amend that (with mostly historical and not so helpful info), notmuch
used to have the ability to sync the "deleted" tag with the T
("trashed") maildir flag (with maildir.synchronize_flags option
set). Other mail clients or offlineimap were then able to delete the
mails locally and/or on a server. However, this too had some issues
(concerning multiple files with the same message-id) making it
potentially dangerous, and was removed [1]. Whether this feature ever
makes a comeback depends on someone tackling the problems. And taking
into account the fact that current users of the deleted tag probably do
not expect the files to be actually deleted.

[1] http://git.notmuchmail.org/git/notmuch/commit/2c262042ac174d7bc96d6035ab9c88bd0abe7f35

BR,
Jani.

Re: Message deletion wisdom

[Antoine Beaupré]

[-- Attachment #1.1: Type: text/plain, Size: 1533 bytes --]

On Tue, 03 Apr 2012 16:32:04 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> So in summary, if you would like to "delete" messages, you can:
> 
>  * add a key binding to emacs (or your favorite ui) to add a "deleted"
>    tag to messages that you want to delete:
> 
>     (define-key notmuch-show-mode-map "d"
>       (lambda ()
>         (interactive)
>         (notmuch-show-tag-message "+deleted")))

Thank you for this. I had tried to reroll your patches
(id:"1326826969-23545-1-git-send-email-jrollins@finestructure.net") on
top of 0.12 and that was a miserable failure, so the above works
well. In fact, I have made it like this instead:

(define-key notmuch-show-mode-map "d"
  (lambda ()
    (interactive)
    (notmuch-show-tag-message "+deleted")
    (notmuch-show-next-open-message)))

... but it doesn't seem to actually go to the next message... oh well,
at least I can delete mail.

Also note that you can delete whole threads with this:

(define-key notmuch-search-mode-map "d"
  (lambda ()
    (interactive)
    (notmuch-search-tag-thread "+deleted")
    (notmuch-search-next-thread)))

... and I have added an undelete function:

(define-key notmuch-search-mode-map "u" 
  (lambda ()
    (interactive)
    (notmuch-search-tag-thread "-deleted")
    (notmuch-search-next-thread)))

>  * and if you really want them purged from disk, delete them manually
>    with:
> 
>     notmuch search --output=files tag:deleted | xargs -l rm

I use this script:


[-- Attachment #1.2: notmuch-purge --]
[-- Type: application/x-sh, Size: 220 bytes --]

[-- Attachment #1.3: Type: text/plain, Size: 613 bytes --]


Finally, I want to voice that I feel a "delete" key, even if it doesn't
delete mails, seems like an important part of a mail user
agent. Archiving mail is one thing, but for the love and respect of
sysadmins and the infrastructure they maintain, please consider adding
at least a way to *tag* those deleted emails.

Having the above keys being defined as standard in notmuch don't seem
like much to ask.

This may be a dissenting view here, but your mail is not that
important. :P

Cheers,

A.
-- 
L'art n'est pas un bureau d'anthropométrie.
                        - Léo Ferré, "Préface"

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Message deletion wisdom

[Antoine Beaupré]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

On Wed, 04 Apr 2012 07:38:37 +0000, Jani Nikula <jani@nikula.org> wrote:
> To amend that (with mostly historical and not so helpful info), notmuch
> used to have the ability to sync the "deleted" tag with the T
> ("trashed") maildir flag (with maildir.synchronize_flags option
> set). Other mail clients or offlineimap were then able to delete the
> mails locally and/or on a server. However, this too had some issues
> (concerning multiple files with the same message-id) making it
> potentially dangerous, and was removed [1]. Whether this feature ever
> makes a comeback depends on someone tackling the problems. And taking
> into account the fact that current users of the deleted tag probably do
> not expect the files to be actually deleted.

I rolled patches for this in the past, see the following message:

id:"1310874973-28437-1-git-send-email-anarcat@koumbit.org"
id:"1310874973-28437-2-git-send-email-anarcat@koumbit.org"

They were indeed refused by the community, but I felt they were still
useful.

I do not use those patches nowadays, however, but to me they would fit
in a broader architectural view of notmuch, where tags are not only
metadata sitting outside of your email, but is also written to the
messages themselves.

The idea is that you can then run notmuch from multiple places without
having to worry about synchronising your tags manually: the tags would
sit in the messages themselves.

A.
-- 
Ce que les siècles des grands abatoirs nous aura appris
Devrait être inscrit au fond de toutes les écoles;
Voici l'homme: le destructeur des mondes est arrivé.
                        - [no one is innocent]

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Message deletion wisdom

[Mark Anderson]

On Thu, 5 Apr 2012 12:20:43 -0400, Antoine Beaupré <anarcat@anarcat.ath.cx> wrote:
> Finally, I want to voice that I feel a "delete" key, even if it doesn't
> delete mails, seems like an important part of a mail user
> agent. Archiving mail is one thing, but for the love and respect of
> sysadmins and the infrastructure they maintain, please consider adding
> at least a way to *tag* those deleted emails.
> 
> Having the above keys being defined as standard in notmuch don't seem
> like much to ask.
> 
> This may be a dissenting view here, but your mail is not that
> important. :P

Hear! Hear!

I also would like to have some natural way to mark things as "I never
expect to look at this again".  Do I really need to keep track of every
vacation and calendar notice?  Or every logfile I have my infrastructure
email to me, that doesn't really need to be in email, but was easily
shoehorned into the existing notification/logging side effects of mail?
My infrastructure is building new models for unit-level testing daily, I
don't need to keep track of every model for years, and I have been using
notmuch for years. Hmm, December 2009, it has been a while. :)

I do set up a deleted tag for my own use, but it would be nice if that
were viewed as a little more natural use case by the software from the
./configure script.  Yes Virginia, notmuch comes with NotMuch of a box.

Thanks,
-Mark

Re: Message deletion wisdom

[Sebastian Spaeth]

Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>So in summary, if you would like to "delete" messages, you can:
>
> * add a key binding to emacs (or your favorite ui) to add a "deleted"
>   tag to messages that you want to delete:
>
>    (define-key notmuch-show-mode-map "d"
>      (lambda ()
>        (interactive)
>        (notmuch-show-tag-message "+deleted")))
>
> * add exclusion for messages with the "deleted" tag:
>
>    notmuch config set search.exclude_tags deleted
>
> * and if you really want them purged from disk, delete them manually
>   with:
>
>    notmuch search --output=files tag:deleted | xargs -l rm

Thanks Jamie, that ws a very helpful summary.

Sebastian

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Re: Message deletion wisdom

[green]

[-- Attachment #1: Type: text/plain, Size: 1385 bytes --]

Mark Anderson wrote at 2012-04-06 15:17 -0500:
> On Thu, 5 Apr 2012 12:20:43 -0400, Antoine Beaupré <anarcat@anarcat.ath.cx> wrote:
> > Finally, I want to voice that I feel a "delete" key, even if it doesn't
> > delete mails, seems like an important part of a mail user
> > agent. Archiving mail is one thing, but for the love and respect of
> > sysadmins and the infrastructure they maintain, please consider adding
> > at least a way to *tag* those deleted emails.
> > 
> > Having the above keys being defined as standard in notmuch don't seem
> > like much to ask.
> > 
> > This may be a dissenting view here, but your mail is not that
> > important. :P
> 
> Hear! Hear!

I agree also.  I receive lots of mail that I never want to see again, nor do 
I want to see it still existing anywhere in my /home/user or in my backups.  
Someone might argue "but what if you decide you want it later" but I can use 
an Expires header to mark a message like "delete in 2 months", so I still 
have a chance to retrieve it within an amount of time I choose.

Somewhat related is the concept of saving spam messages.  I can understand 
the reasoning behind training bayesian filters, but I have never kept spam.  
I switched to bogofilter recently, and it was correctly filtering mail very 
quickly, perhaps within a few weeks (I did train it using non-spam messages).

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]