update on delete/rename support

update on delete/rename support

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]

Hey, folks.  I was wondering if there was an update on getting support
for mail deleting and moving/renaming into notmuch.  I wanted to try to
push on it a little bit since I believe that support for deleting/moving
messages is the most important outstanding issue at the moment.

I finally got around to really trying notmuch (xmas present to myself)
and I'm finding the inability to delete/move messages quite problematic.
Currently, my inbox consists of a local maildir that is sync'd to a
remote IMAP mailbox via offlineimap.  As I go through my inbox I either
move mail to an archive, or delete unwanted messages.  A system like
this obviously requires that my MUA supports moving and deleting
messages.  I know a lot of people have similar setups, and in fact some
have brought it up on this list.  Probably the main issues here is that
remote IMAP stores generally do not have unlimited space, and a lot of
people use remote IMAP stores.  Even for those that can just pull all
their mail directly, local mail stores are generally not unlimited
either, and the ability to completely purge messages is still important.

It seems to me (without really knowing what's involved) that for notmuch
the main issue is really handling deleted messages.  Notmuch handles new
mail fine (obviously) and moved/renamed messages are really just
messages that are deleted while an identical but differently named new
message shows up somewhere else, right?  Would someone who understands
what's going on under the hood be up for giving a brief explanation
about what the issues involved are?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: update on delete/rename support

[David Bremner]

On Sat, 26 Dec 2009 00:21:15 -0500, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> Hey, folks.  I was wondering if there was an update on getting support
> for mail deleting and moving/renaming into notmuch.  I wanted to try to
> push on it a little bit since I believe that support for deleting/moving
> messages is the most important outstanding issue at the moment.

As far as I know Carl is working away on this.

In the meantime, I am using Mikhail Gusarov's patch

    id:1259788526-14205-1-git-send-email-dottedmag@dottedmag.net

It is not perfect performance-wise, and iirc, it only deals with
renames, but it does let me use notmuch as my main mail reader.  You may
have to wrap your "notmuch new" call in a "notmuch save", "notmuch
restore" pair if you discover things coming back to your inbox. This is
less onerous if you use my "lazy restore" patch.

    id:1260310063-11731-1-git-send-email-david@tethera.net

The whole dump-maildirsync-new-restore cycle takes about 30s for me
check mail, a bit longer if there is actually new mail.

David

Re: update on delete/rename support

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1323 bytes --]

On Sat, 26 Dec 2009 12:12:09 -0400, David Bremner <bremner@unb.ca> wrote:
> On Sat, 26 Dec 2009 00:21:15 -0500, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > Hey, folks.  I was wondering if there was an update on getting support
> > for mail deleting and moving/renaming into notmuch.  I wanted to try to
> > push on it a little bit since I believe that support for deleting/moving
> > messages is the most important outstanding issue at the moment.
> 
> As far as I know Carl is working away on this.

Yes. I've got a series of about 20 commits reworking the internals a
bit, and then one patch past that that I still need to break down into
several clean commits.

I think I've got things working now to cleanly detect file deletion. And
the only bug that I *know* is still there is a performance bug, (it's
somehow spending time doing something silly with known messages when it
should just march along and only do real work with new and deleted
files).

Anyway, I hope to have something to actually publish early next
week. Please forgive me for not sharing more as I go along here. The
patches change the storage within the database, so one could be left in
a very unpleasant state if using patches that don't match the database
storage that we'll be using in the end.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: update on delete/rename support

[micah anderson]

On Sat, 26 Dec 2009 12:12:09 -0400, David Bremner <bremner@unb.ca> wrote:

>You may have to wrap your "notmuch new" call in a "notmuch save",
>"notmuch restore" pair if you discover things coming back to your inbox.

do you mean "notmuch dump" here instead of "notmuch save"?

> The whole dump-maildirsync-new-restore cycle takes about 30s for me
> check mail, a bit longer if there is actually new mail.

I'm trying to unpack what you are saying here. Are you saying that if
you are getting things coming back into your inbox, and you have not
applied your "lazy restore" patch, then you would need a script to check
your email which does:

notmuch dump > /tmp/notmuch.dump
offlineimap 
notmuch new
notmuch restore < /tmp/notmuch.dump

micah

Re: update on delete/rename support

[David Bremner]

On Sat, 26 Dec 2009 14:20:38 -0500, micah anderson <micah@riseup.net> wrote:
> 
> >You may have to wrap your "notmuch new" call in a "notmuch save",
> >"notmuch restore" pair if you discover things coming back to your
> >inbox
> 
> do you mean "notmuch dump" here instead of "notmuch save"?

Yes, sorry.


> I'm trying to unpack what you are saying here. Are you saying that if
> you are getting things coming back into your inbox, and you have not
> applied your "lazy restore" patch, then you would need a script to check
> your email which does:
> 
> notmuch dump > /tmp/notmuch.dump
> offlineimap 
> notmuch new
> notmuch restore < /tmp/notmuch.dump

The script is the same in both cases, but without my patch, it takes
more like 10 minutes for me.

d

Re: update on delete/rename support

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

On Sat, Dec 26, 2009 at 10:06:24AM -0800, Carl Worth wrote:
> Yes. I've got a series of about 20 commits reworking the internals a
> bit, and then one patch past that that I still need to break down into
> several clean commits.

Carl, that's really great to hear.  Thanks so much for the response.
I'm definitely looking forward to seeing what you've got in store, and
finally moving full-fledged to notmuch.  I'm happy to be a guinea pig
tester for your changes as well.

Hopefully we can add some emacs key bindings for deleting and moving
messages as well.

jamie.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]