unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [PATCH 0/5] Store message modification times in the DB
@ 2011-12-13 17:11 Thomas Jost
  2011-12-13 17:11 ` [PATCH 1/5] Fix comments about what is stored in the database Thomas Jost
                   ` (5 more replies)
  0 siblings, 6 replies; 17+ messages in thread
From: Thomas Jost @ 2011-12-13 17:11 UTC (permalink / raw)
  To: notmuch

Hello world,

This is a patch series I've been working on for some time in order to be
able to sync my tags on several computers. I'm posting it now, but
please consider it as a RFC rather than something that is ready to be
pushed.

The basic idea is to the last time each message was modified, i.e. "the
message was added to the DB", "a tag was added" or "a tag was removed".

This mtime is accessible through a library function and in the JSON
output of "notmuch show". It is also searchable with the "mtime:" prefix
and with timestamp ranges, like for searching messages by date:

    notmuch search mtime:$(date +%s 2011-12-01)..$(date +%s)

This can then be used in scripts or helper programs to do incremental
dumps or tags synchronization. (I already have a script to do
incremental backups, but it needs some cleaning, and I'm still working
on something for sync'ing tags, but it's starting to work really well;
I'll post them later).

This can be seen as an alternative to David Bremner's jlog branch, but
with several differences:

+ no external dependency
+ everything is stored in the notmuch DB: atomicity for free!
- when a message is removed we lose everything about it, which makes the
  sync process more complicated
- for a human, it's harder to manipulate timestamps than log messages
- this can store much less data than a proper log system

On IRC amdragon suggested using a simple sequence number instead of a
timestamp. This would indeed eliminate the need for proper time
synchronization between computers one would want to keep in sync, and it
would reduce the risk of time-going-backward problems, but IMHO it would
cause more problems: no global clock --> no simple way to tell if DB #A
is more recent than DB #B.

So, here are the patches:
- first a little fix to the comments describing the DB schema (not
  specific to this patch series at all, I just noticed it when rebasing
  this series)
- the second commit adds the MTIME value to the database schema, and
  creates the functions used to update and access this value.
- the third commit makes the MTIME value searchable with a range syntax.
- the fourth commit adds the MTIME to the JSON output of "notmuch show".
- the fifth and last commit adds Message.get_mtime() to the Python
  bindings.

Please tell me what you think of this.

Best regards,
Thomas

Thomas Jost (5):
  Fix comments about what is stored in the database
  lib: Add a MTIME value to every mail document
  lib: Make MTIME values searchable
  show: include mtime in JSON output
  python: add get_mtime() to the Message class

 bindings/python/notmuch/message.py |   20 ++++++++++++++++++++
 lib/database-private.h             |    1 +
 lib/database.cc                    |   14 +++++++++++++-
 lib/message.cc                     |   32 ++++++++++++++++++++++++++++++++
 lib/notmuch-private.h              |    6 +++++-
 lib/notmuch.h                      |    4 ++++
 notmuch-show.c                     |    7 ++++---
 notmuch.1                          |   14 ++++++++++++--
 notmuch.c                          |   13 ++++++++++---
 9 files changed, 101 insertions(+), 10 deletions(-)

-- 
1.7.8

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2012-01-02 14:57 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-12-13 17:11 [PATCH 0/5] Store message modification times in the DB Thomas Jost
2011-12-13 17:11 ` [PATCH 1/5] Fix comments about what is stored in the database Thomas Jost
2011-12-23 19:10   ` David Bremner
2011-12-13 17:11 ` [PATCH 2/5] lib: Add a MTIME value to every mail document Thomas Jost
2011-12-14 21:54   ` Mark Anderson
2011-12-21  0:34     ` Thomas Jost
2011-12-15  0:45   ` Austin Clements
2011-12-21  1:00     ` Thomas Jost
2011-12-13 17:11 ` [PATCH 3/5] lib: Make MTIME values searchable Thomas Jost
2011-12-13 17:11 ` [PATCH 4/5] show: include mtime in JSON output Thomas Jost
2011-12-13 17:11 ` [PATCH 5/5] python: add get_mtime() to the Message class Thomas Jost
2012-01-02 14:56   ` Sebastian Spaeth
2011-12-19 16:34 ` [PATCH 0/5] Store message modification times in the DB David Edmondson
2011-12-19 19:48   ` Austin Clements
2011-12-19 22:56     ` Tom Prince
2011-12-20  8:32     ` David Edmondson
2011-12-20 15:05       ` Austin Clements

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).