* notmuch and imap [musing, no code :)]
@ 2009-12-15 12:05 David Bremner
2009-12-16 0:54 ` Alec Berryman
0 siblings, 1 reply; 5+ messages in thread
From: David Bremner @ 2009-12-15 12:05 UTC (permalink / raw
To: notmuch
Recent discussions on IRC (I forget with whom, sorry), brought me back
to thinking about syncing notmuch with imap. In addition to the flags
\Seen, \Answered, \Draft, \Deleted, and \Flagged, imap servers can
optionally support user defined keywords (i.e. tags). At least courier
and dovecot do. These keywords are imap "atoms", which, without tracing
though the BNF [1] completely look like they can can have (ascii)
alphanumeric, and punctuation other than brackets, quotes and "%", "*".
So the mapping is relatively nice between notmuch tags and imap
keywords.
One idea I had was to extend some imap syncing program to sync to
notmuch. mbsync (confusingly also named isync) is a C based one. It is
indeed relatively easy to add a new backend; I made a new "notmuch"
driver for mbsync in an hour or so that is actually just a maildir
driver. I am a little discouraged by some of mbsync code (there are
lots of places with a buffer hard-coded to size 16, with a comment to
change that later when keyword support is added), but in principle, I
think this could work. A more fundamental issue is that mbsync, like
most similar programs uses an index file to keep track of the sync
state, and it seems somehow wrong (or at least fragile) to keep a second
database with essentially the same information in it.
To write a custom sync program would require some imap code; this is
somewhere between trivial and not; mbsync's imap driver is about 1900
lines of C. There is the uw c-client library; I remember some security
issues but that was more than a decade ago. There are some newer
libraries like tinymail and libetpan, but they seem to have a whole
bunch of stuff not needed for notmuch.
d
[1]: http://www.faqs.org/rfcs/rfc3501.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: notmuch and imap [musing, no code :)]
2009-12-15 12:05 notmuch and imap [musing, no code :)] David Bremner
@ 2009-12-16 0:54 ` Alec Berryman
2009-12-16 13:18 ` David Bremner
0 siblings, 1 reply; 5+ messages in thread
From: Alec Berryman @ 2009-12-16 0:54 UTC (permalink / raw
To: David Bremner; +Cc: notmuch
David Bremner on 2009-12-15 08:05:13 -0400:
> Recent discussions on IRC (I forget with whom, sorry), brought me back
> to thinking about syncing notmuch with imap. In addition to the flags
> \Seen, \Answered, \Draft, \Deleted, and \Flagged, imap servers can
> optionally support user defined keywords (i.e. tags). At least courier
> and dovecot do. These keywords are imap "atoms", which, without tracing
> though the BNF [1] completely look like they can can have (ascii)
> alphanumeric, and punctuation other than brackets, quotes and "%", "*".
I am also very interested in syncing tags between computers. I started
implementing storage of tags in message headers. It's has been done
before - several mutt extensions and systems use X-Label. This approach
offers several advantages to notmuch dump/restore and IMAP flags:
- compatability: most mail clients can search on headers, so even if
you're not using notmuch full-time (squirrelmail? phone?), you can
get some benefit from it
- works with offlineimap without further effort: no new transport
mechanism required
- a migration path: text-based mail sorting tools like procmail can
easily set headers
- backups are easy: you can't miss backing up your tags because
they're in the messages
There are security concerns (need to strip incoming messages of tags so
no one tags your mail for you), privacy concerns (if you forward the
entire message as an attachment, may want to strip tags), and space
concerns (how many flags?), but I think they can be worked around.
I haven't gotten very far with my implementation due to time
constraints. It reads tags fine, but I haven't implemented writing,
which is the involved part. I hope to get to this between Christmas and
New Years, but who knows.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: notmuch and imap [musing, no code :)]
2009-12-16 0:54 ` Alec Berryman
@ 2009-12-16 13:18 ` David Bremner
2009-12-16 14:29 ` Marten Veldthuis
2010-01-08 3:00 ` martin f krafft
0 siblings, 2 replies; 5+ messages in thread
From: David Bremner @ 2009-12-16 13:18 UTC (permalink / raw
To: Alec Berryman; +Cc: notmuch
On Tue, 15 Dec 2009 19:54:11 -0500, Alec Berryman <alec@thened.net> wrote:
> There are security concerns (need to strip incoming messages of tags so
> no one tags your mail for you), privacy concerns (if you forward the
> entire message as an attachment, may want to strip tags), and space
> concerns (how many flags?), but I think they can be worked around.
I agree that the labels-in-headers approach has some nice advantages. I
haven't thought through merging of tag lists, but maybe that is no worse
than other approaches. One thing that worries me a bit is that notmuch
updates tags often, and each of these updates would require rewriting
the message, at least in the obvious implementation. I'd hate to finally
have Xapian issue 350 fixed, only to take an equivalent hit by rewriting
the message. Maybe it is not that slow.
d
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: notmuch and imap [musing, no code :)]
2009-12-16 13:18 ` David Bremner
@ 2009-12-16 14:29 ` Marten Veldthuis
2010-01-08 3:00 ` martin f krafft
1 sibling, 0 replies; 5+ messages in thread
From: Marten Veldthuis @ 2009-12-16 14:29 UTC (permalink / raw
To: David Bremner, Alec Berryman; +Cc: notmuch
On Wed, 16 Dec 2009 09:18:16 -0400, David Bremner <david@tethera.net> wrote:
> I agree that the labels-in-headers approach has some nice advantages. I
> haven't thought through merging of tag lists, but maybe that is no worse
> than other approaches. One thing that worries me a bit is that notmuch
> updates tags often, and each of these updates would require rewriting
> the message, at least in the obvious implementation. I'd hate to finally
> have Xapian issue 350 fixed, only to take an equivalent hit by rewriting
> the message. Maybe it is not that slow.
If it is, maybe we could do the sync-back by renaming notmuch-new to
notmuch-sync, and doing it there.
--
- Marten
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: notmuch and imap [musing, no code :)]
2009-12-16 13:18 ` David Bremner
2009-12-16 14:29 ` Marten Veldthuis
@ 2010-01-08 3:00 ` martin f krafft
1 sibling, 0 replies; 5+ messages in thread
From: martin f krafft @ 2010-01-08 3:00 UTC (permalink / raw
Cc: notmuch
[-- Attachment #1: Type: text/plain, Size: 926 bytes --]
also sprach David Bremner <david@tethera.net> [2009.12.17.0218 +1300]:
> I agree that the labels-in-headers approach has some nice
> advantages. I haven't thought through merging of tag lists, but
> maybe that is no worse than other approaches. One thing that
> worries me a bit is that notmuch updates tags often, and each of
> these updates would require rewriting the message, at least in the
> obvious implementation.
If we separated implicit and explicit tags, then that issue would
not exist, and from all I can tell, only the privacy issues, and the
risk of losing mail when files are rewritten persist.
--
martin | http://madduck.net/ | http://two.sentenc.es/
DISCLAIMER: this entire message is privileged communication, intended
for the sole use of its recipients only. If you read it even though
you know you aren't supposed to, you're a poopy-head.
spamtraps: madduck.bogus@madduck.net
[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-01-08 3:00 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-12-15 12:05 notmuch and imap [musing, no code :)] David Bremner
2009-12-16 0:54 ` Alec Berryman
2009-12-16 13:18 ` David Bremner
2009-12-16 14:29 ` Marten Veldthuis
2010-01-08 3:00 ` martin f krafft
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).