[PATCH] notmuch new: add a --in-directory option

[PATCH] notmuch new: add a --in-directory option

[Mark Walters]

This patch adds a --in-directory=folder option to notmuch new which
tells it to only check for new messages inside folder (relative to the
database root)
---

NOTE This is only very lightly tested (but seems to work) so please
make sure you backup the notmuch database before testing!


 notmuch-new.c    |   10 +++++++++-
 test/T050-new.sh |   15 +++++++++++++++
 2 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/notmuch-new.c b/notmuch-new.c
index 8529fdd..d3526ac 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -922,6 +922,8 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
     notmuch_bool_t timer_is_active = FALSE;
     notmuch_bool_t no_hooks = FALSE;
     notmuch_bool_t quiet = FALSE, verbose = FALSE;
+    char *directory = NULL;
+    char *path = NULL;
 
     add_files_state.verbosity = VERBOSITY_NORMAL;
     add_files_state.debug = FALSE;
@@ -932,6 +934,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
 	{ NOTMUCH_OPT_BOOLEAN,  &verbose, "verbose", 'v', 0 },
 	{ NOTMUCH_OPT_BOOLEAN,  &add_files_state.debug, "debug", 'd', 0 },
 	{ NOTMUCH_OPT_BOOLEAN,  &no_hooks, "no-hooks", 'n', 0 },
+	{ NOTMUCH_OPT_STRING, &directory, "in-directory", 'i', 0  },
 	{ 0, 0, 0, 0, 0 }
     };
 
@@ -1021,7 +1024,12 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
 	timer_is_active = TRUE;
     }
 
-    ret = add_files (notmuch, db_path, &add_files_state);
+    if (directory)
+	path = talloc_asprintf (config, "%s/%s", db_path, directory);
+    else
+	path = talloc_strdup (config, db_path);
+
+    ret = add_files (notmuch, path, &add_files_state);
     if (ret)
 	goto DONE;
 
diff --git a/test/T050-new.sh b/test/T050-new.sh
index b7668ff..b8e68a3 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -263,4 +263,19 @@ notmuch search --format=text0 --output=files --offset=1 --limit=1 '*' | xargs -0
 output=$(NOTMUCH_NEW --quiet)
 test_expect_equal "$output" ""
 
+test_begin_subtest "In-directory"
+rm -rf "${MAIL_DIR}"/* "${MAIL_DIR}"/.notmuch
+mkdir "${MAIL_DIR}"/def
+mkdir "${MAIL_DIR}"/ghi
+generate_message [dir]=def
+generate_message [dir]=ghi
+generate_message
+
+output=$(NOTMUCH_NEW --in-directory=def)
+test_expect_equal "$output" "Added 1 new message to the database."
+
+test_begin_subtest "New after --in-directory"
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "Added 2 new messages to the database."
+
 test_done
-- 
1.7.9.1

Re: [PATCH] notmuch new: add a --in-directory option

[Austin Clements]

I haven't thought about this as a patch yet, but wanted to point out
that it should probably skip the removal step if it's only scanning a
subdirectory.  Otherwise, messages that are moved out of the scanned
directory into some other may be considered deleted and get removed
from the database.

That aside, I'm curious what the use case for this is.

Quoth Mark Walters on Mar 06 at  8:58 pm:
> This patch adds a --in-directory=folder option to notmuch new which
> tells it to only check for new messages inside folder (relative to the
> database root)
> ---
> 
> NOTE This is only very lightly tested (but seems to work) so please
> make sure you backup the notmuch database before testing!
> 
> 
>  notmuch-new.c    |   10 +++++++++-
>  test/T050-new.sh |   15 +++++++++++++++
>  2 files changed, 24 insertions(+), 1 deletions(-)
> 
> diff --git a/notmuch-new.c b/notmuch-new.c
> index 8529fdd..d3526ac 100644
> --- a/notmuch-new.c
> +++ b/notmuch-new.c
> @@ -922,6 +922,8 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
>      notmuch_bool_t timer_is_active = FALSE;
>      notmuch_bool_t no_hooks = FALSE;
>      notmuch_bool_t quiet = FALSE, verbose = FALSE;
> +    char *directory = NULL;
> +    char *path = NULL;
>  
>      add_files_state.verbosity = VERBOSITY_NORMAL;
>      add_files_state.debug = FALSE;
> @@ -932,6 +934,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
>  	{ NOTMUCH_OPT_BOOLEAN,  &verbose, "verbose", 'v', 0 },
>  	{ NOTMUCH_OPT_BOOLEAN,  &add_files_state.debug, "debug", 'd', 0 },
>  	{ NOTMUCH_OPT_BOOLEAN,  &no_hooks, "no-hooks", 'n', 0 },
> +	{ NOTMUCH_OPT_STRING, &directory, "in-directory", 'i', 0  },
>  	{ 0, 0, 0, 0, 0 }
>      };
>  
> @@ -1021,7 +1024,12 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
>  	timer_is_active = TRUE;
>      }
>  
> -    ret = add_files (notmuch, db_path, &add_files_state);
> +    if (directory)
> +	path = talloc_asprintf (config, "%s/%s", db_path, directory);
> +    else
> +	path = talloc_strdup (config, db_path);
> +
> +    ret = add_files (notmuch, path, &add_files_state);
>      if (ret)
>  	goto DONE;
>  
> diff --git a/test/T050-new.sh b/test/T050-new.sh
> index b7668ff..b8e68a3 100755
> --- a/test/T050-new.sh
> +++ b/test/T050-new.sh
> @@ -263,4 +263,19 @@ notmuch search --format=text0 --output=files --offset=1 --limit=1 '*' | xargs -0
>  output=$(NOTMUCH_NEW --quiet)
>  test_expect_equal "$output" ""
>  
> +test_begin_subtest "In-directory"
> +rm -rf "${MAIL_DIR}"/* "${MAIL_DIR}"/.notmuch
> +mkdir "${MAIL_DIR}"/def
> +mkdir "${MAIL_DIR}"/ghi
> +generate_message [dir]=def
> +generate_message [dir]=ghi
> +generate_message
> +
> +output=$(NOTMUCH_NEW --in-directory=def)
> +test_expect_equal "$output" "Added 1 new message to the database."
> +
> +test_begin_subtest "New after --in-directory"
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "Added 2 new messages to the database."
> +
>  test_done

Re: [PATCH] notmuch new: add a --in-directory option

[Adam Wolfe Gordon]

On Thu, Mar 6, 2014 at 2:06 PM, Austin Clements <amdragon@mit.edu> wrote:
> That aside, I'm curious what the use case for this is.

My usecase for this is the same as for the similar patch I worked on
previously (id:1373762746-22308-1-git-send-email-awg+notmuch@xvx.ca):

I use inotify to watch for new messages in my maildir. When I receive
a new message, I run notmuch new (after waiting a couple of seconds
since messages sometimes arrive in batches). But since I already know
exactly what changed, there's no need for notmuch new to scan the
whole directory tree. I'm not sure whether this patch lets me specify
a file, or only a directory: the former would be preferable for my
usecase, but either way it's probably an enhancement for me.

-- Adam

Re: [PATCH] notmuch new: add a --in-directory option

[Tomi Ollila]

On Thu, Mar 06 2014, Adam Wolfe Gordon <awg+notmuch@xvx.ca> wrote:

> On Thu, Mar 6, 2014 at 2:06 PM, Austin Clements <amdragon@mit.edu> wrote:
>> That aside, I'm curious what the use case for this is.

I am also interested... How are mails delivered so that
--in-directory=folder is useful

>
> My usecase for this is the same as for the similar patch I worked on
> previously (id:1373762746-22308-1-git-send-email-awg+notmuch@xvx.ca):

... as this is (much more) interesting to me too ...


> I use inotify to watch for new messages in my maildir. When I receive
> a new message, I run notmuch new (after waiting a couple of seconds
> since messages sometimes arrive in batches). But since I already know
> exactly what changed, there's no need for notmuch new to scan the
> whole directory tree. I'm not sure whether this patch lets me specify
> a file, or only a directory: the former would be preferable for my
> usecase, but either way it's probably an enhancement for me.

my mails are delivered by
https://github.com/domo141/nottoomuch/blob/master/md5mda.sh
to ~/mail/received/??/??????????????????????????????
and log to the deliveries are kept:

$ echo; tail -5 mail/log/md5mda-201403.log

2014-03-07 (Fri) 11:58:37: Added 'received/a8/5bb1949aa85bb84287499555dabb00'
2014-03-07 (Fri) 12:33:57: Added 'received/34/b786dabccc980b3b651e85f3d7fd48'
2014-03-07 (Fri) 12:35:01: Added 'received/c0/e89f83383d066cc922512dc5eddb5d'
2014-03-07 (Fri) 13:04:32: Added 'received/f5/ff8b132d3c4f8ca63215d8de41de9c'
2014-03-07 (Fri) 13:25:00: Added 'received/9a/848dd40274fe5b6d6bcbad80b2e7fd'

From that information I could launch notmuch new [files...] 
(or notmuch new --batch !) to index all files since last attempt(*)


Tomi


(*) routine would be: 
	1) get size of last log file (for final stored offset)
        2) read the info of last read log file and offset
        3) feed the filenames between these 2 offsets.
        4) update the final stored offset
        5) check whether the size (or name) of the last log 
	   file has changed and if had, go step 2

> -- Adam

Tomi