unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* v4 merged config.
@ 2021-01-16 17:03 David Bremner
  2021-01-16 17:03 ` [PATCH 01/38] lib: add _notmuch_string_map_set David Bremner
                   ` (37 more replies)
  0 siblings, 38 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch

This obsoletes [1]. Interdiff is below.  The main changes are dealing
with hooks in new locations, and some documentation. For a summary, of
changes, see the new documentation, in patch 38/38.


[1] id:20210103233547.122707-1-david@tethera.net



 doc/man1/notmuch-config.rst |   43 +++++++--
 doc/man5/notmuch-hooks.rst  |    6 -
 hooks.c                     |    7 -
 lib/config.cc               |    5 +
 lib/notmuch.h               |    3 
 lib/open.cc                 |   42 +++++++++
 notmuch-client.h            |    2 
 notmuch-count.c             |    2 
 notmuch-insert.c            |    7 -
 notmuch-new.c               |    4 
 test/T140-excludes.sh       |   15 +++
 test/T400-hooks.sh          |  197 ++++++++++++++++++++++++--------------------
 test/T590-libconfig.sh      |   32 +++++++
 test/T750-gzip.sh           |    2 
 test/test-lib.el            |    4 
 15 files changed, 258 insertions(+), 113 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 769f336a..bc597957 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -17,10 +17,6 @@ DESCRIPTION
 The **config** command can be used to get or set settings in the notmuch
 configuration file and corresponding database.
 
-Items marked **[STORED IN DATABASE]** are only in the database.  They
-should not be placed in the configuration file, and should be accessed
-programmatically as described in the SYNOPSIS above.
-
 **get**
     The value of the specified configuration item is printed to
     stdout. If the item has multiple values (it is a list), each value
@@ -54,6 +50,11 @@ The available configuration items are described below.
 
     Default: ``$MAILDIR`` variable if set, otherwise ``$HOME/mail``.
 
+**database.hook_dir**
+
+    Directory containing hooks run by notmuch commands. See
+    **notmuch-hooks(5)**.
+
 **user.name**
     Your full name.
 
@@ -134,7 +135,7 @@ The available configuration items are described below.
 
     Default: ``true``.
 
-**index.decrypt** **[STORED IN DATABASE]**
+**index.decrypt**
     Policy for decrypting encrypted messages during indexing.  Must be
     one of: ``false``, ``auto``, ``nostash``, or ``true``.
 
@@ -187,7 +188,7 @@ The available configuration items are described below.
 
     Default: ``auto``.
 
-**index.header.<prefix>** **[STORED IN DATABASE]**
+**index.header.<prefix>**
     Define the query prefix <prefix>, based on a mail header. For
     example ``index.header.List=List-Id`` will add a probabilistic
     prefix ``List:`` that searches the ``List-Id`` field.  User
@@ -202,7 +203,7 @@ The available configuration items are described below.
     (since notmuch 0.30, "compact" and "field_processor" are
     always included.)
 
-**query.<name>** **[STORED IN DATABASE]**
+**query.<name>**
     Expansion for named query called <name>. See
     **notmuch-search-terms(7)** for more information about named
     queries.
@@ -214,8 +215,32 @@ The following environment variables can be used to control the behavior
 of notmuch.
 
 **NOTMUCH\_CONFIG**
-    Specifies the location of the notmuch configuration file. Notmuch
-    will use ${HOME}/.notmuch-config if this variable is not set.
+    Specifies the location of the notmuch configuration file.
+
+**NOTMUCH_PROFILE**
+    Selects among notmuch configurations.
+
+FILES
+=====
+
+CONFIGURATION
+-------------
+
+If ``NOTMUCH_CONFIG`` is unset, notmuch tries (in order)
+
+- ``$XDG_CONFIG_HOME/notmuch/<profile>/config`` where ``<profile>`` is
+  defined by ``$NOTMUCH_PROFILE`` or "default"
+- ``${HOME}/.notmuch-config<profile>`` where ``<profile>`` is
+  ``.$NOTMUCH_PROFILE`` or ""
+
+Hooks
+-----
+
+If ``database.hook_dir`` is unset, notmuch tries (in order)
+
+- ``$XDG_CONFIG_HOME/notmuch/<profile>/hooks`` where ``<profile>`` is
+  defined by ``$NOTMUCH_PROFILE`` or "default"
+- ``<database.path>/.notmuch/hooks``
 
 SEE ALSO
 ========
diff --git a/doc/man5/notmuch-hooks.rst b/doc/man5/notmuch-hooks.rst
index de2ed0c2..c509afb3 100644
--- a/doc/man5/notmuch-hooks.rst
+++ b/doc/man5/notmuch-hooks.rst
@@ -5,15 +5,15 @@ notmuch-hooks
 SYNOPSIS
 ========
 
-$DATABASEDIR/.notmuch/hooks/*
+<hook_dir>/{pre-new, post-new, post-insert}
 
 DESCRIPTION
 ===========
 
 Hooks are scripts (or arbitrary executables or symlinks to such) that
 notmuch invokes before and after certain actions. These scripts reside
-in the .notmuch/hooks directory within the database directory and must
-have executable permissions.
+in a directory defined as described in **notmuch-config(1)**. They
+must have executable permissions.
 
 The currently available hooks are described below.
 
diff --git a/hooks.c b/hooks.c
index 59c58070..ec89b22e 100644
--- a/hooks.c
+++ b/hooks.c
@@ -24,14 +24,15 @@
 #include <sys/wait.h>
 
 int
-notmuch_run_hook (const char *db_path, const char *hook)
+notmuch_run_hook (notmuch_database_t *notmuch, const char *hook)
 {
     char *hook_path;
     int status = 0;
     pid_t pid;
 
-    hook_path = talloc_asprintf (NULL, "%s/%s/%s/%s", db_path, ".notmuch",
-				 "hooks", hook);
+    hook_path = talloc_asprintf (notmuch, "%s/%s",
+				 notmuch_config_get (notmuch, NOTMUCH_CONFIG_HOOK_DIR),
+				 hook);
     if (hook_path == NULL) {
 	fprintf (stderr, "Out of memory\n");
 	return 1;
diff --git a/lib/config.cc b/lib/config.cc
index 7a1494be..4b115a07 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -303,6 +303,8 @@ notmuch_config_values_get (notmuch_config_values_t *values) {
 
 void
 notmuch_config_values_start (notmuch_config_values_t *values) {
+    if (values == NULL)
+	return;
     if (values->children) {
 	talloc_free (values->children);
     }
@@ -388,6 +390,8 @@ _notmuch_config_key_to_string (notmuch_config_key_t key) {
     switch (key) {
     case NOTMUCH_CONFIG_DATABASE_PATH:
 	return "database.path";
+    case NOTMUCH_CONFIG_HOOK_DIR:
+	return "database.hook_dir";
     case NOTMUCH_CONFIG_EXCLUDE_TAGS:
 	return "search.exclude_tags";
     case NOTMUCH_CONFIG_NEW_TAGS:
@@ -426,6 +430,7 @@ _notmuch_config_default (void *ctx, notmuch_config_key_t key) {
 	return "inbox;unread";
     case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS:
 	return "true";
+    case NOTMUCH_CONFIG_HOOK_DIR:
     case NOTMUCH_CONFIG_NEW_IGNORE:
     case NOTMUCH_CONFIG_USER_NAME:
     case NOTMUCH_CONFIG_PRIMARY_EMAIL:
diff --git a/lib/notmuch.h b/lib/notmuch.h
index e51b738d..3c3fd487 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2461,6 +2461,7 @@ notmuch_config_list_destroy (notmuch_config_list_t *config_list);
 typedef enum _notmuch_config_key {
     NOTMUCH_CONFIG_FIRST,
     NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST,
+    NOTMUCH_CONFIG_HOOK_DIR,
     NOTMUCH_CONFIG_EXCLUDE_TAGS,
     NOTMUCH_CONFIG_NEW_TAGS,
     NOTMUCH_CONFIG_NEW_IGNORE,
@@ -2562,7 +2563,7 @@ notmuch_config_values_move_to_next (notmuch_config_values_t *values);
 /**
  * reset the 'values' iterator to the first element
  *
- * @param[in,out] values iterator
+ * @param[in,out] values iterator. A NULL value is ignored.
  *
  * @since libnotmuch 5.4 (notmuch 0.32)
  *
diff --git a/lib/open.cc b/lib/open.cc
index 92661271..eb7c8a01 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -67,6 +67,44 @@ _xdg_dir (void *ctx,
 			    profile_name);
 }
 
+static notmuch_status_t
+_choose_hook_dir (notmuch_database_t *notmuch,
+		  const char *profile,
+		  char **message)
+{
+    const char *config;
+    const char *hook_dir;
+    struct stat st;
+    int err;
+
+    hook_dir = notmuch_config_get (notmuch, NOTMUCH_CONFIG_HOOK_DIR);
+
+    if (hook_dir)
+	return NOTMUCH_STATUS_SUCCESS;
+
+    config = _xdg_dir (notmuch, "XDG_CONFIG_HOME", ".config", profile);
+    if (! config)
+	return  NOTMUCH_STATUS_PATH_ERROR;
+
+    hook_dir = talloc_asprintf (notmuch, "%s/hooks", config);
+
+    err = stat (hook_dir, &st);
+    if (err) {
+	if (errno == ENOENT) {
+	    const char *database_path = notmuch_database_get_path (notmuch);
+	    hook_dir = talloc_asprintf (notmuch, "%s/.notmuch/hooks", database_path);
+	} else {
+	    IGNORE_RESULT (asprintf (message, "Error: Cannot stat %s: %s.\n",
+				     hook_dir, strerror (errno)));
+	    return NOTMUCH_STATUS_FILE_ERROR;
+	}
+    }
+
+    _notmuch_config_cache (notmuch, NOTMUCH_CONFIG_HOOK_DIR, hook_dir);
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static notmuch_status_t
 _load_key_file (const char *path,
 		const char *profile,
@@ -300,6 +338,10 @@ notmuch_database_open_with_config (const char *database_path,
 	if (status)
 	    goto DONE;
 
+	status = _choose_hook_dir (notmuch, profile, &message);
+	if (status)
+	    goto DONE;
+
 	status = _notmuch_config_load_defaults (notmuch);
 	if (status)
 	    goto DONE;
diff --git a/notmuch-client.h b/notmuch-client.h
index 9e09c36a..f60f5406 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -339,7 +339,7 @@ const char *
 _notmuch_config_get_path (notmuch_config_t *config);
 
 int
-notmuch_run_hook (const char *db_path, const char *hook);
+notmuch_run_hook (notmuch_database_t *notmuch, const char *hook);
 
 bool
 debugger_is_active (void);
diff --git a/notmuch-count.c b/notmuch-count.c
index 321c9207..048b1f44 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -80,7 +80,7 @@ print_count (notmuch_database_t *notmuch, const char *query_str,
 	return -1;
     }
 
-    for (;
+    for (notmuch_config_values_start (exclude_tags);
 	 notmuch_config_values_valid (exclude_tags);
 	 notmuch_config_values_move_to_next (exclude_tags)) {
 
diff --git a/notmuch-insert.c b/notmuch-insert.c
index e483b949..0f272e2e 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -481,7 +481,6 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
     notmuch_process_shared_options (argv[0]);
 
 
-    /* XXX TODO replace this use of DATABASE_PATH with something specific to hooks */
     db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH);
 
     if (! db_path)
@@ -570,7 +569,7 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
     status = add_file (notmuch, newpath, tag_ops, synchronize_flags, keep, indexing_cli_choices.opts);
 
     /* Commit changes. */
-    close_status = notmuch_database_destroy (notmuch);
+    close_status = notmuch_database_close (notmuch);
     if (close_status) {
 	/* Hold on to the first error, if any. */
 	if (! status)
@@ -595,9 +594,11 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
 
     if (hooks && status == NOTMUCH_STATUS_SUCCESS) {
 	/* Ignore hook failures. */
-	notmuch_run_hook (db_path, "post-insert");
+	notmuch_run_hook (notmuch, "post-insert");
     }
 
+    notmuch_database_destroy (notmuch);
+
     talloc_free (local);
 
     return status_to_exit (status);
diff --git a/notmuch-new.c b/notmuch-new.c
index 0f416939..2fc34e2c 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -1167,7 +1167,7 @@ notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmu
     }
 
     if (hooks) {
-	ret = notmuch_run_hook (db_path, "pre-new");
+	ret = notmuch_run_hook (notmuch, "pre-new");
 	if (ret)
 	    return EXIT_FAILURE;
     }
@@ -1284,7 +1284,7 @@ notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmu
     notmuch_database_close (notmuch);
 
     if (hooks && ! ret && ! interrupted)
-	ret = notmuch_run_hook (db_path, "post-new");
+	ret = notmuch_run_hook (notmuch, "post-new");
 
     notmuch_database_destroy (notmuch);
 
diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index cef07095..0cacc41d 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -286,6 +286,21 @@ test_begin_subtest "Count, default exclusion: tag in query (threads)"
 output=$(notmuch count --output=threads tag:test and tag:deleted)
 test_expect_equal "$output" "3"
 
+test_begin_subtest "Count, default exclusion, batch"
+notmuch count  --batch --output=messages<<EOF > OUTPUT
+tag:test
+tag:test and tag:deleted
+tag:test
+tag:test and tag:deleted
+EOF
+cat <<EOF >EXPECTED
+2
+4
+2
+4
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Count, exclude=true: tag in query (messages)"
 output=$(notmuch count --exclude=true tag:test and tag:deleted)
 test_expect_equal "$output" "4"
diff --git a/test/T400-hooks.sh b/test/T400-hooks.sh
index 49c690eb..b9894853 100755
--- a/test/T400-hooks.sh
+++ b/test/T400-hooks.sh
@@ -2,8 +2,6 @@
 test_description='hooks'
 . $(dirname "$0")/test-lib.sh || exit 1
 
-HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
-
 create_echo_hook () {
     local TOKEN="${RANDOM}"
     mkdir -p ${HOOK_DIR}
@@ -16,6 +14,7 @@ EOF
 }
 
 create_failing_hook () {
+    local HOOK_DIR=${2}
     mkdir -p ${HOOK_DIR}
     cat <<EOF >"${HOOK_DIR}/${1}"
 #!/bin/sh
@@ -24,98 +23,120 @@ EOF
     chmod +x "${HOOK_DIR}/${1}"
 }
 
-rm_hooks () {
-    rm -rf ${HOOK_DIR}
-}
-
 # add a message to generate mail dir and database
 add_message
 # create maildir structure for notmuch-insert
 mkdir -p "$MAIL_DIR"/{cur,new,tmp}
 
-test_begin_subtest "pre-new is run"
-rm_hooks
-generate_message
-create_echo_hook "pre-new" expected output
-notmuch new > /dev/null
-test_expect_equal_file expected output
-
-test_begin_subtest "post-new is run"
-rm_hooks
-generate_message
-create_echo_hook "post-new" expected output
-notmuch new > /dev/null
-test_expect_equal_file expected output
-
-test_begin_subtest "post-insert hook is run"
-rm_hooks
-generate_message
-create_echo_hook "post-insert" expected output
-notmuch insert < "$gen_msg_filename"
-test_expect_equal_file expected output
-
-test_begin_subtest "pre-new is run before post-new"
-rm_hooks
-generate_message
-create_echo_hook "pre-new" pre-new.expected pre-new.output
-create_echo_hook "post-new" post-new.expected post-new.output
-notmuch new > /dev/null
-test_expect_equal_file post-new.expected post-new.output
-
-test_begin_subtest "pre-new non-zero exit status (hook status)"
-rm_hooks
-generate_message
-create_failing_hook "pre-new"
-output=`notmuch new 2>&1`
-test_expect_equal "$output" "Error: pre-new hook failed with status 13"
-
-# depends on the previous subtest leaving broken hook behind
-test_begin_subtest "pre-new non-zero exit status (notmuch status)"
-test_expect_code 1 "notmuch new"
-
-# depends on the previous subtests leaving 1 new message behind
-test_begin_subtest "pre-new non-zero exit status aborts new"
-rm_hooks
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "Added 1 new message to the database."
-
-test_begin_subtest "post-new non-zero exit status (hook status)"
-rm_hooks
-generate_message
-create_failing_hook "post-new"
-NOTMUCH_NEW 2>output.stderr >output
-cat output.stderr >> output
-echo "Added 1 new message to the database." > expected
-echo "Error: post-new hook failed with status 13" >> expected
-test_expect_equal_file expected output
-
-# depends on the previous subtest leaving broken hook behind
-test_begin_subtest "post-new non-zero exit status (notmuch status)"
-test_expect_code 1 "notmuch new"
-
-test_begin_subtest "post-insert hook does not affect insert status"
-rm_hooks
-generate_message
-create_failing_hook "post-insert"
-test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null"
-
-test_begin_subtest "hook without executable permissions"
-rm_hooks
-mkdir -p ${HOOK_DIR}
-cat <<EOF >"${HOOK_DIR}/pre-new"
-#!/bin/sh
-echo foo
+for config in traditional profile explicit XDG; do
+    unset NOTMUCH_PROFILE
+    notmuch config set database.hook_dir
+    case $config in
+	traditional)
+	    HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
+	    ;;
+	profile)
+	    dir=${HOME}/.config/notmuch/other
+	    mkdir -p ${dir}
+	    HOOK_DIR=${dir}/hooks
+	    cp ${NOTMUCH_CONFIG} ${dir}/config
+	    export NOTMUCH_PROFILE=other
+	    ;;
+	explicit)
+	    HOOK_DIR=${HOME}/.notmuch-hooks
+	    mkdir -p $HOOK_DIR
+	    notmuch config set database.hook_dir $HOOK_DIR
+	    ;;
+	*)
+	    HOOK_DIR=${HOME}/.config/notmuch/default/hooks
+	    ;;
+    esac
+
+    test_begin_subtest "pre-new is run [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "pre-new" expected output $HOOK_DIR
+    notmuch new > /dev/null
+    test_expect_equal_file expected output
+
+    test_begin_subtest "post-new is run [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "post-new" expected output $HOOK_DIR
+    notmuch new > /dev/null
+    test_expect_equal_file expected output
+
+    test_begin_subtest "post-insert hook is run [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "post-insert" expected output $HOOK_DIR
+    notmuch insert < "$gen_msg_filename"
+    test_expect_equal_file expected output
+
+    test_begin_subtest "pre-new is run before post-new [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "pre-new" pre-new.expected pre-new.output $HOOK_DIR
+    create_echo_hook "post-new" post-new.expected post-new.output $HOOK_DIR
+    notmuch new > /dev/null
+    test_expect_equal_file post-new.expected post-new.output
+
+    test_begin_subtest "pre-new non-zero exit status (hook status) [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_failing_hook "pre-new" $HOOK_DIR
+    output=`notmuch new 2>&1`
+    test_expect_equal "$output" "Error: pre-new hook failed with status 13"
+
+    # depends on the previous subtest leaving broken hook behind
+    test_begin_subtest "pre-new non-zero exit status (notmuch status) [${config}]"
+    test_expect_code 1 "notmuch new"
+
+    # depends on the previous subtests leaving 1 new message behind
+    test_begin_subtest "pre-new non-zero exit status aborts new [${config}]"
+    rm -rf ${HOOK_DIR}
+    output=$(NOTMUCH_NEW)
+    test_expect_equal "$output" "Added 1 new message to the database."
+
+    test_begin_subtest "post-new non-zero exit status (hook status) [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_failing_hook "post-new" $HOOK_DIR
+    NOTMUCH_NEW 2>output.stderr >output
+    cat output.stderr >> output
+    echo "Added 1 new message to the database." > expected
+    echo "Error: post-new hook failed with status 13" >> expected
+    test_expect_equal_file expected output
+
+    # depends on the previous subtest leaving broken hook behind
+    test_begin_subtest "post-new non-zero exit status (notmuch status) [${config}]"
+    test_expect_code 1 "notmuch new"
+
+    test_begin_subtest "post-insert hook does not affect insert status [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_failing_hook "post-insert" $HOOK_DIR
+    test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null"
+
+    test_begin_subtest "hook without executable permissions [${config}]"
+    rm -rf ${HOOK_DIR}
+    mkdir -p ${HOOK_DIR}
+    cat <<EOF >"${HOOK_DIR}/pre-new"
+    #!/bin/sh
+    echo foo
 EOF
-output=`notmuch new 2>&1`
-test_expect_code 1 "notmuch new"
-
-test_begin_subtest "hook execution failure"
-rm_hooks
-mkdir -p ${HOOK_DIR}
-cat <<EOF >"${HOOK_DIR}/pre-new"
-no hashbang, execl fails
+    output=`notmuch new 2>&1`
+    test_expect_code 1 "notmuch new"
+
+    test_begin_subtest "hook execution failure [${config}]"
+    rm -rf ${HOOK_DIR}
+    mkdir -p ${HOOK_DIR}
+    cat <<EOF >"${HOOK_DIR}/pre-new"
+    no hashbang, execl fails
 EOF
-chmod +x "${HOOK_DIR}/pre-new"
-test_expect_code 1 "notmuch new"
+    chmod +x "${HOOK_DIR}/pre-new"
+    test_expect_code 1 "notmuch new"
 
+    rm -rf ${HOOK_DIR}
+done
 test_done
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 97f8fdc7..c78ed204 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -201,6 +201,37 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+test_begin_subtest "NOTMUCH_CONFIG_HOOK_DIR: traditional"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+   const char *val = notmuch_config_get (db, NOTMUCH_CONFIG_HOOK_DIR);
+   printf("database.hook_dir = %s\n", val);
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+database.hook_dir = MAIL_DIR/.notmuch/hooks
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "NOTMUCH_CONFIG_HOOK_DIR: xdg"
+dir="${HOME}/.config/notmuch/default/hooks"
+mkdir -p $dir
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+   const char *val = notmuch_config_get (db, NOTMUCH_CONFIG_HOOK_DIR);
+   printf("database.hook_dir = %s\n", val);
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+database.hook_dir = CWD/home/.config/notmuch/default/hooks
+== stderr ==
+EOF
+rmdir $dir
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "notmuch_config_get_values"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
@@ -333,6 +364,7 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 MAIL_DIR
+MAIL_DIR/.notmuch/hooks
 
 inbox;unread
 NULL
diff --git a/test/T750-gzip.sh b/test/T750-gzip.sh
index 807086fd..7d550e66 100755
--- a/test/T750-gzip.sh
+++ b/test/T750-gzip.sh
@@ -171,7 +171,7 @@ test_expect_equal_file EXPECTED OUTPUT
 add_email_corpus lkml
 test_begin_subtest "new doesn't run out of file descriptors with many gzipped files"
 ulimit -n 200
-find ${MAIL_DIR} -name .notmuch -prune -false -o  -type f  -exec gzip --recursive {} \;
+find ${MAIL_DIR} -name .notmuch -prune -o  -type f -print0 | xargs -0 gzip --
 test_expect_success "notmuch new"
 
 test_done
diff --git a/test/test-lib.el b/test/test-lib.el
index ec16c59c..4de5b292 100644
--- a/test/test-lib.el
+++ b/test/test-lib.el
@@ -1,4 +1,4 @@
-;; test-lib.el --- auxiliary stuff for Notmuch Emacs tests.
+;;; test-lib.el --- auxiliary stuff for Notmuch Emacs tests
 ;;
 ;; Copyright © Carl Worth
 ;; Copyright © David Edmondson
@@ -20,6 +20,8 @@
 ;;
 ;; Authors: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
 
+;;; Code:
+
 (require 'cl-lib)
 
 ;; Ensure that the dynamic variables that are defined by this library\r

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 01/38] lib: add _notmuch_string_map_set
  2021-01-16 17:03 v4 merged config David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-24 11:28   ` Tomi Ollila
  2021-01-16 17:03 ` [PATCH 02/38] lib: cache configuration information from database David Bremner
                   ` (36 subsequent siblings)
  37 siblings, 1 reply; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will be used (and tested) by the configuration caching code to be
added in the next commit.
---
 lib/notmuch-private.h |  5 +++++
 lib/string-map.c      | 15 +++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 57ec7f72..51016b0b 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -638,6 +638,11 @@ _notmuch_string_map_append (notmuch_string_map_t *map,
 			    const char *key,
 			    const char *value);
 
+void
+_notmuch_string_map_set (notmuch_string_map_t *map,
+			 const char *key,
+			 const char *value);
+
 const char *
 _notmuch_string_map_get (notmuch_string_map_t *map, const char *key);
 
diff --git a/lib/string-map.c b/lib/string-map.c
index a88404c7..1f3215fb 100644
--- a/lib/string-map.c
+++ b/lib/string-map.c
@@ -142,6 +142,21 @@ bsearch_first (notmuch_string_pair_t *array, size_t len, const char *key, bool e
 	return NULL;
 
 }
+void
+_notmuch_string_map_set (notmuch_string_map_t *map, const char *key, const char *val)
+{
+    notmuch_string_pair_t *pair;
+
+    /* this means that calling string_map_set invalidates iterators */
+    _notmuch_string_map_sort (map);
+    pair = bsearch_first (map->pairs, map->length, key, true);
+    if (! pair)
+       _notmuch_string_map_append (map, key, val);
+    else {
+       talloc_free (pair->value);
+       pair->value = talloc_strdup (map->pairs, val);
+    }
+}
 
 const char *
 _notmuch_string_map_get (notmuch_string_map_t *map, const char *key)
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 02/38] lib: cache configuration information from database
  2021-01-16 17:03 v4 merged config David Bremner
  2021-01-16 17:03 ` [PATCH 01/38] lib: add _notmuch_string_map_set David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 03/38] lib: add stub for notmuch_database_open_with_config David Bremner
                   ` (35 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The main goal is to allow configuration information to be temporarily
overridden by a separate config file. That will require further
changes not in this commit.

The performance impact is unclear, and will depend on the balance
between number of queries and number of distinct metadata items read
on the first call to n_d_get_config.
---
 lib/config.cc             | 58 ++++++++++++++++++++++++++++++++++-----
 lib/database-private.h    |  3 ++
 lib/notmuch-private.h     |  3 ++
 lib/open.cc               |  7 ++++-
 lib/prefix.cc             | 18 ++++++------
 test/T562-lib-database.sh |  7 ++---
 6 files changed, 75 insertions(+), 21 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index 0b760dbc..c079d752 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -50,6 +50,11 @@ notmuch_database_set_config (notmuch_database_t *notmuch,
     if (status)
 	return status;
 
+    if (! notmuch->config) {
+	if ((status = _notmuch_config_load_from_database (notmuch)))
+	    return status;
+    }
+
     try {
 	notmuch->writable_xapian_db->set_metadata (CONFIG_PREFIX + key, value);
     } catch (const Xapian::Error &error) {
@@ -58,7 +63,13 @@ notmuch_database_set_config (notmuch_database_t *notmuch,
 	_notmuch_database_log (notmuch, "Error: A Xapian exception occurred setting metadata: %s\n",
 			       error.get_msg ().c_str ());
     }
-    return status;
+
+    if (status)
+	return status;
+
+    _notmuch_string_map_set (notmuch->config, key, value);
+
+    return NOTMUCH_STATUS_SUCCESS;
 }
 
 static notmuch_status_t
@@ -84,17 +95,25 @@ notmuch_database_get_config (notmuch_database_t *notmuch,
 			     const char *key,
 			     char **value)
 {
-    std::string strval;
+    const char* stored_val;
     notmuch_status_t status;
 
+    if (! notmuch->config) {
+	if ((status = _notmuch_config_load_from_database (notmuch)))
+	    return status;
+    }
+
     if (! value)
 	return NOTMUCH_STATUS_NULL_POINTER;
 
-    status = _metadata_value (notmuch, key, strval);
-    if (status)
-	return status;
-
-    *value = strdup (strval.c_str ());
+    stored_val = _notmuch_string_map_get (notmuch->config, key);
+    if (! stored_val) {
+	/* XXX in principle this API should be fixed so empty string
+	 * is distinguished from not found */
+	*value = strdup("");
+    } else {
+	*value = strdup (stored_val);
+    }
 
     return NOTMUCH_STATUS_SUCCESS;
 }
@@ -201,3 +220,28 @@ notmuch_config_list_destroy (notmuch_config_list_t *list)
 {
     talloc_free (list);
 }
+
+notmuch_status_t
+_notmuch_config_load_from_database (notmuch_database_t *notmuch)
+{
+    notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+    notmuch_config_list_t *list;
+
+    if (notmuch->config == NULL)
+	notmuch->config = _notmuch_string_map_create (notmuch);
+
+    if (unlikely(notmuch->config == NULL))
+	return NOTMUCH_STATUS_OUT_OF_MEMORY;
+
+    status = notmuch_database_get_config_list (notmuch, "", &list);
+    if (status)
+	return status;
+
+    for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) {
+	_notmuch_string_map_append (notmuch->config,
+				    notmuch_config_list_key (list),
+				    notmuch_config_list_value (list));
+    }
+
+    return status;
+}
diff --git a/lib/database-private.h b/lib/database-private.h
index c9bc712b..d83cf0d0 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -228,6 +228,9 @@ struct _notmuch_database {
      * here, but at least they are small */
     notmuch_string_map_t *user_prefix;
     notmuch_string_map_t *user_header;
+
+    /* Cached and possibly overridden configuration */
+    notmuch_string_map_t *config;
 };
 
 /* Prior to database version 3, features were implied by the database
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 51016b0b..969caac0 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -704,6 +704,9 @@ struct _notmuch_indexopts {
 
 #define EMPTY_STRING(s) ((s)[0] == '\0')
 
+/* config.cc */
+notmuch_status_t
+_notmuch_config_load_from_database (notmuch_database_t * db);
 NOTMUCH_END_DECLS
 
 #ifdef __cplusplus
diff --git a/lib/open.cc b/lib/open.cc
index 1b17e63a..0001794a 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -90,7 +90,7 @@ notmuch_database_open_verbose (const char *path,
     notmuch->exception_reported = false;
     notmuch->status_string = NULL;
     notmuch->path = talloc_strdup (notmuch, path);
-
+    notmuch->config = NULL;
     strip_trailing (notmuch->path, '/');
 
     notmuch->writable_xapian_db = NULL;
@@ -180,6 +180,11 @@ notmuch_database_open_verbose (const char *path,
 	notmuch->query_parser->add_rangeprocessor (notmuch->date_range_processor);
 	notmuch->query_parser->add_rangeprocessor (notmuch->last_mod_range_processor);
 
+	/* Configuration information is needed to set up query parser */
+	status = _notmuch_config_load_from_database (notmuch);
+	if (status)
+	    goto DONE;
+
 	status = _notmuch_database_setup_standard_query_fields (notmuch);
 	if (status)
 	    goto DONE;
diff --git a/lib/prefix.cc b/lib/prefix.cc
index dd7b193d..71a76991 100644
--- a/lib/prefix.cc
+++ b/lib/prefix.cc
@@ -166,8 +166,7 @@ _notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch)
 notmuch_status_t
 _notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch)
 {
-    notmuch_config_list_t *list;
-    notmuch_status_t status;
+    notmuch_string_map_iterator_t *list;
 
     notmuch->user_prefix = _notmuch_string_map_create (notmuch);
     if (notmuch->user_prefix == NULL)
@@ -177,15 +176,16 @@ _notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch)
     if (notmuch->user_header == NULL)
 	return NOTMUCH_STATUS_OUT_OF_MEMORY;
 
-    status = notmuch_database_get_config_list (notmuch, CONFIG_HEADER_PREFIX, &list);
-    if (status)
-	return status;
+    list = _notmuch_string_map_iterator_create (notmuch->config, CONFIG_HEADER_PREFIX, FALSE);
+    if (! list)
+	INTERNAL_ERROR ("unable to read headers from configuration");
 
-    for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) {
+    for (; _notmuch_string_map_iterator_valid (list);
+	 _notmuch_string_map_iterator_move_to_next (list)) {
 
 	prefix_t query_field;
 
-	const char *key = notmuch_config_list_key (list)
+	const char *key = _notmuch_string_map_iterator_key (list)
 			  + sizeof (CONFIG_HEADER_PREFIX) - 1;
 
 	_notmuch_string_map_append (notmuch->user_prefix,
@@ -194,7 +194,7 @@ _notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch)
 
 	_notmuch_string_map_append (notmuch->user_header,
 				    key,
-				    notmuch_config_list_value (list));
+				    _notmuch_string_map_iterator_value (list));
 
 	query_field.name = talloc_strdup (notmuch, key);
 	query_field.prefix = _user_prefix (notmuch, key);
@@ -204,7 +204,7 @@ _notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch)
 	_setup_query_field_default (&query_field, notmuch);
     }
 
-    notmuch_config_list_destroy (list);
+    _notmuch_string_map_iterator_destroy (list);
 
     return NOTMUCH_STATUS_SUCCESS;
 }
diff --git a/test/T562-lib-database.sh b/test/T562-lib-database.sh
index dd4f2566..887851dd 100755
--- a/test/T562-lib-database.sh
+++ b/test/T562-lib-database.sh
@@ -340,7 +340,7 @@ test_expect_equal_file EXPECTED OUTPUT
 test_begin_subtest "get config from closed database"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
-        const char *result;
+        char *result;
         EXPECT0(notmuch_database_close (db));
         stat = notmuch_database_get_config (db, "foo", &result);
         printf("%d\n",  stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION);
@@ -348,9 +348,8 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 EOF
 cat <<EOF > EXPECTED
 == stdout ==
-1
+0
 == stderr ==
-Error: A Xapian exception occurred getting metadata: Database has been closed
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
@@ -376,7 +375,7 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
         notmuch_indexopts_t *result;
         EXPECT0(notmuch_database_close (db));
         result = notmuch_database_get_default_indexopts (db);
-        printf("%d\n",  result == NULL);
+        printf("%d\n", result != NULL);
     }
 EOF
 cat <<EOF > EXPECTED
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 03/38] lib: add stub for notmuch_database_open_with_config
  2021-01-16 17:03 v4 merged config David Bremner
  2021-01-16 17:03 ` [PATCH 01/38] lib: add _notmuch_string_map_set David Bremner
  2021-01-16 17:03 ` [PATCH 02/38] lib: cache configuration information from database David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 04/38] lib/open: add support for config profiles and default locations David Bremner
                   ` (34 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Initially document the intended API and copy the code from
notmuch_database_open_verbose. Most of the documented functionality is
not there yet.
---
 lib/config.cc             |  34 +++++++++
 lib/notmuch-private.h     |   3 +
 lib/notmuch.h             | 140 ++++++++++++++++++++++++++++++--------
 lib/open.cc               |  42 ++++++++++--
 test/T562-lib-database.sh |   4 +-
 test/T590-libconfig.sh    |  39 +++++++++--
 6 files changed, 221 insertions(+), 41 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index c079d752..8bce7ba8 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -245,3 +245,37 @@ _notmuch_config_load_from_database (notmuch_database_t *notmuch)
 
     return status;
 }
+
+notmuch_status_t
+_notmuch_config_load_from_file (notmuch_database_t *notmuch,
+				GKeyFile *file)
+{
+    notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+    gchar **groups,**keys, *val;
+
+    if (notmuch->config == NULL)
+	notmuch->config = _notmuch_string_map_create (notmuch);
+
+    if (unlikely(notmuch->config == NULL)) {
+	status = NOTMUCH_STATUS_OUT_OF_MEMORY;
+	goto DONE;
+    }
+
+    for (groups = g_key_file_get_groups (file, NULL); *groups; groups++) {
+	for (keys = g_key_file_get_keys (file, *groups, NULL, NULL); *keys; keys++) {
+	    char *absolute_key = talloc_asprintf(notmuch, "%s.%s", *groups,  *keys);
+	    val = g_key_file_get_value (file, *groups, *keys, NULL);
+	    if (! val) {
+		status = NOTMUCH_STATUS_FILE_ERROR;
+		goto DONE;
+	    }
+	    _notmuch_string_map_set (notmuch->config, absolute_key, val);
+	    talloc_free (absolute_key);
+	    if (status)
+		goto DONE;
+	}
+    }
+
+ DONE:
+    return status;
+}
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 969caac0..40b1a855 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -707,6 +707,9 @@ struct _notmuch_indexopts {
 /* config.cc */
 notmuch_status_t
 _notmuch_config_load_from_database (notmuch_database_t * db);
+
+notmuch_status_t
+_notmuch_config_load_from_file (notmuch_database_t * db, GKeyFile *file);
 NOTMUCH_END_DECLS
 
 #ifdef __cplusplus
diff --git a/lib/notmuch.h b/lib/notmuch.h
index c66e78b1..9346640a 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -301,52 +301,138 @@ typedef enum {
 } notmuch_database_mode_t;
 
 /**
- * Open an existing notmuch database located at 'path'.
+ * Deprecated alias for notmuch_database_open_with_config with
+ * config_path=error_message=NULL
+ * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
+ */
+/* NOTMUCH_DEPRECATED(5, 4) */
+notmuch_status_t
+notmuch_database_open (const char *path,
+		       notmuch_database_mode_t mode,
+		       notmuch_database_t **database);
+/**
+ * Deprecated alias for notmuch_database_open_with_config with
+ * config_path=NULL
+ *
+ * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
+ *
+ */
+/* NOTMUCH_DEPRECATED(5, 4) */
+notmuch_status_t
+notmuch_database_open_verbose (const char *path,
+			       notmuch_database_mode_t mode,
+			       notmuch_database_t **database,
+			       char **error_message);
+
+/**
+ * Open an existing notmuch database located at 'database_path', using
+ * configuration in 'config_path'.
+ *
+ * @param[in]	database_path
+ * @parblock
+ * Path to existing database.
+ *
+ * A notmuch database is a Xapian database containing appropriate
+ * metadata.
  *
  * The database should have been created at some time in the past,
  * (not necessarily by this process), by calling
- * notmuch_database_create with 'path'. By default the database should be
- * opened for reading only. In order to write to the database you need to
- * pass the NOTMUCH_DATABASE_MODE_READ_WRITE mode.
+ * notmuch_database_create.
+ *
+ * If 'database_path' is NULL, use the location specified
+ *
+ * - in the environment variable NOTMUCH_DATABASE, if non-empty
+ *
+ * - in a configuration file, located as described under 'config_path'
+ *
+ * - by $XDG_DATA_HOME/notmuch/$PROFILE where XDG_DATA_HOME defaults
+ *   to "$HOME/.local/share" and PROFILE as as discussed in
+ *   'profile'
+ *
+ * If 'database_path' is non-NULL, but does not appear to be a Xapian
+ * database, check for a directory '.notmuch/xapian' below
+ * 'database_path' (this is the behavior of
+ * notmuch_database_open_verbose pre-0.32).
+ *
+ * @endparblock
+ * @param[in]	mode
+ * @parblock
+ * Mode to open database. Use one of #NOTMUCH_DATABASE_MODE_READ_WRITE
+ * or #NOTMUCH_DATABASE_MODE_READ_ONLY
+ *
+ * @endparblock
+ * @param[in]  config_path
+ * @parblock
+ * Path to config file.
+ *
+ * Config file is key-value, with mandatory sections. See
+ * <em>notmuch-config(5)</em> for more information. The key-value pair
+ * overrides the corresponding configuration data stored in the
+ * database (see <em>notmuch_database_get_config</em>)
  *
- * An existing notmuch database can be identified by the presence of a
- * directory named ".notmuch" below 'path'.
+ * If <em>config_path</em> is NULL use the path specified
+ *
+ * - in environment variable <em>NOTMUCH_CONFIG</em>, if non-empty
+ *
+ * - by  <em>XDG_CONFIG_HOME</em>/notmuch/ where
+ *   XDG_CONFIG_HOME defaults to "$HOME/.config".
+ *
+ * - by $HOME/.notmuch-config
+ *
+ * If <em>config_path</em> is "" (empty string) then do not
+ * open any configuration file.
+ * @endparblock
+ * @param[in] profile:
+ * @parblock
+ * Name of profile (configuration/database variant).
+ *
+ * If non-NULL, append to the directory / file path determined for
+ * <em>config_path</em> and <em>database_path</em>.
+ *
+ * If NULL then use
+ * - environment variable NOTMUCH_PROFILE if defined,
+ * - otherwise "default" for directories and "" (empty string) for paths.
+ *
+ * @endparblock
+ * @param[out] database
+ * @parblock
+ * Pointer to database object. May not be NULL.
  *
  * The caller should call notmuch_database_destroy when finished with
  * this database.
  *
  * In case of any failure, this function returns an error status and
- * sets *database to NULL (after printing an error message on stderr).
+ * sets *database to NULL.
  *
- * Return value:
+ * @endparblock
+ * @param[out] error_message
+ * If non-NULL, store error message from opening the database.
+ * Any such message is allocated by \a malloc(3) and should be freed
+ * by the caller.
  *
- * NOTMUCH_STATUS_SUCCESS: Successfully opened the database.
+ * @retval NOTMUCH_STATUS_SUCCESS: Successfully opened the database.
  *
- * NOTMUCH_STATUS_NULL_POINTER: The given 'path' argument is NULL.
+ * @retval NOTMUCH_STATUS_NULL_POINTER: The given \a database
+ * argument is NULL.
  *
- * NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory.
+ * @retval NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory.
  *
- * NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the
- *	database file (such as permission denied, or file not found,
+ * @retval NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the
+ *	database or config file (such as permission denied, or file not found,
  *	etc.), or the database version is unknown.
  *
- * NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred.
- */
-notmuch_status_t
-notmuch_database_open (const char *path,
-		       notmuch_database_mode_t mode,
-		       notmuch_database_t **database);
-/**
- * Like notmuch_database_open, except optionally return an error
- * message. This message is allocated by malloc and should be freed by
- * the caller.
+ * @retval NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred.
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
  */
 
 notmuch_status_t
-notmuch_database_open_verbose (const char *path,
-			       notmuch_database_mode_t mode,
-			       notmuch_database_t **database,
-			       char **error_message);
+notmuch_database_open_with_config (const char *database_path,
+				   notmuch_database_mode_t mode,
+				   const char *config_path,
+				   const char *profile,
+				   notmuch_database_t **database,
+				   char **error_message);
 
 /**
  * Retrieve last status string for given database.
diff --git a/lib/open.cc b/lib/open.cc
index 0001794a..7acaea7b 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -32,30 +32,57 @@ notmuch_database_open_verbose (const char *path,
 			       notmuch_database_mode_t mode,
 			       notmuch_database_t **database,
 			       char **status_string)
+{
+    return notmuch_database_open_with_config (path, mode, "", NULL,
+					      database, status_string);
+}
+
+notmuch_status_t
+notmuch_database_open_with_config (const char *database_path,
+				   notmuch_database_mode_t mode,
+				   const char *config_path,
+				   unused(const char *profile),
+				   notmuch_database_t **database,
+				   char **status_string)
 {
     notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
     void *local = talloc_new (NULL);
     notmuch_database_t *notmuch = NULL;
     char *notmuch_path, *xapian_path, *incompat_features;
+    char *configured_database_path = NULL;
     char *message = NULL;
     struct stat st;
     int err;
     unsigned int version;
+    GKeyFile *key_file = NULL;
     static int initialized = 0;
 
-    if (path == NULL) {
+    /* XXX TODO: default locations for NULL case, handle profiles */
+    if (config_path != NULL && ! EMPTY_STRING (config_path)) {
+	key_file = g_key_file_new ();
+	if (! g_key_file_load_from_file (key_file, config_path, G_KEY_FILE_NONE, NULL)) {
+	    status = NOTMUCH_STATUS_FILE_ERROR;
+	    goto DONE;
+	}
+	configured_database_path = g_key_file_get_value (key_file, "database", "path", NULL);
+    }
+
+    if (database_path == NULL)
+	database_path = configured_database_path;
+
+    if (database_path == NULL) {
 	message = strdup ("Error: Cannot open a database for a NULL path.\n");
 	status = NOTMUCH_STATUS_NULL_POINTER;
 	goto DONE;
     }
 
-    if (path[0] != '/') {
+    if (database_path[0] != '/') {
 	message = strdup ("Error: Database path must be absolute.\n");
 	status = NOTMUCH_STATUS_PATH_ERROR;
 	goto DONE;
     }
 
-    if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) {
+    if (! (notmuch_path = talloc_asprintf (local, "%s/%s", database_path, ".notmuch"))) {
 	message = strdup ("Out of memory\n");
 	status = NOTMUCH_STATUS_OUT_OF_MEMORY;
 	goto DONE;
@@ -89,8 +116,8 @@ notmuch_database_open_verbose (const char *path,
     notmuch = talloc_zero (NULL, notmuch_database_t);
     notmuch->exception_reported = false;
     notmuch->status_string = NULL;
-    notmuch->path = talloc_strdup (notmuch, path);
-    notmuch->config = NULL;
+    notmuch->path = talloc_strdup (notmuch, database_path);
+
     strip_trailing (notmuch->path, '/');
 
     notmuch->writable_xapian_db = NULL;
@@ -185,6 +212,11 @@ notmuch_database_open_verbose (const char *path,
 	if (status)
 	    goto DONE;
 
+	if (key_file)
+	    status = _notmuch_config_load_from_file (notmuch, key_file);
+	if (status)
+	    goto DONE;
+
 	status = _notmuch_database_setup_standard_query_fields (notmuch);
 	if (status)
 	    goto DONE;
diff --git a/test/T562-lib-database.sh b/test/T562-lib-database.sh
index 887851dd..db251fe7 100755
--- a/test/T562-lib-database.sh
+++ b/test/T562-lib-database.sh
@@ -343,12 +343,12 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
         char *result;
         EXPECT0(notmuch_database_close (db));
         stat = notmuch_database_get_config (db, "foo", &result);
-        printf("%d\n",  stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION);
+        printf("%d\n",  stat == NOTMUCH_STATUS_SUCCESS);
     }
 EOF
 cat <<EOF > EXPECTED
 == stdout ==
-0
+1
 == stderr ==
 EOF
 test_expect_equal_file EXPECTED OUTPUT
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 8c34acf9..5fb1bb87 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -16,7 +16,12 @@ int main (int argc, char** argv)
    char *val;
    notmuch_status_t stat;
 
-   EXPECT0(notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_WRITE, &db));
+   EXPECT0(notmuch_database_open_with_config (argv[1],
+                                              NOTMUCH_DATABASE_MODE_READ_WRITE,
+                                              argv[2],
+                                              NULL,
+                                              &db,
+                                              NULL));
 
 EOF
 
@@ -26,7 +31,7 @@ cat <<EOF > c_tail
 EOF
 
 test_begin_subtest "notmuch_database_{set,get}_config"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
    EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1"));
    EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2"));
@@ -46,7 +51,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 
 test_begin_subtest "notmuch_database_get_config_list: empty list"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
    notmuch_config_list_t *list;
    EXPECT0(notmuch_database_get_config_list (db, "nonexistent", &list));
@@ -78,7 +83,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: all pairs"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
    notmuch_config_list_t *list;
    EXPECT0(notmuch_database_set_config (db, "zzzafter", "afterval"));
@@ -123,7 +128,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: one prefix"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
    notmuch_config_list_t *list;
    EXPECT0(notmuch_database_get_config_list (db, "test.key", &list));
@@ -142,7 +147,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "dump config"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
     EXPECT0(notmuch_database_set_config (db, "key with spaces", "value, with, spaces!"));
 }
@@ -160,7 +165,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "restore config"
 notmuch dump --include=config >EXPECTED
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
     EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
 }
@@ -169,4 +174,24 @@ notmuch restore --include=config <EXPECTED
 notmuch dump --include=config >OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+backup_database
+test_begin_subtest "override config from file"
+notmuch config set test.key1 overridden
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+{
+   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
+   printf("test.key2 = %s\n", val);
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 test_done
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 04/38] lib/open: add support for config profiles and default locations
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (2 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 03/38] lib: add stub for notmuch_database_open_with_config David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-24 11:58   ` Tomi Ollila
  2021-01-16 17:03 ` [PATCH 05/38] CLI: generalize notmuch_config_mode_t David Bremner
                   ` (33 subsequent siblings)
  37 siblings, 1 reply; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This commit fills in the remainder of the documented functionality for
n_d_open_with_config with respect to config file location. Similar
searching default locations of the database file still needs to be
added.
---
 lib/open.cc            |  95 +++++++++++++++++++++----
 test/T590-libconfig.sh | 153 ++++++++++++++++++++++++++++++++++++++---
 2 files changed, 227 insertions(+), 21 deletions(-)

diff --git a/lib/open.cc b/lib/open.cc
index 7acaea7b..76255283 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -37,6 +37,82 @@ notmuch_database_open_verbose (const char *path,
 					      database, status_string);
 }
 
+static const char *
+_xdg_dir (void *ctx,
+	  const char *xdg_root_variable,
+	  const char *xdg_prefix,
+	  const char *profile_name)
+{
+    const char *xdg_root = getenv (xdg_root_variable);
+    const char *home = getenv ("HOME");
+
+    if (! xdg_root) {
+	if (! home) return NULL;
+
+	xdg_root = talloc_asprintf (ctx,
+				    "%s/%s",
+				    home,
+				    xdg_prefix);
+    }
+
+    if (! profile_name)
+	profile_name = getenv ("NOTMUCH_PROFILE");
+
+    if (! profile_name)
+	profile_name = "default";
+
+    return talloc_asprintf (ctx,
+			    "%s/notmuch/%s",
+			    xdg_root,
+			    profile_name);
+}
+
+static notmuch_status_t
+_load_key_file (const char *path,
+		const char *profile,
+		GKeyFile **key_file)
+{
+    notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+    void *local = talloc_new (NULL);
+
+    if (path && EMPTY_STRING (path))
+	goto DONE;
+
+    if (! path)
+	path = getenv ("NOTMUCH_CONFIG");
+
+    if (! path) {
+	const char *dir = _xdg_dir (local, "XDG_CONFIG_HOME", ".config", profile);
+
+	if (dir) {
+	    path = talloc_asprintf (local, "%s/config", dir);
+	    if (access (path, R_OK) !=0)
+		path = NULL;
+	}
+    }
+
+    if (! path) {
+	const char *home = getenv ("HOME");
+
+	path = talloc_asprintf (local, "%s/.notmuch-config", home);
+
+	if (! profile)
+	    profile = getenv ("NOTMUCH_PROFILE");
+
+	if (profile)
+	    path = talloc_asprintf (local, "%s.%s", path, profile);
+    }
+
+    *key_file = g_key_file_new ();
+    if (! g_key_file_load_from_file (*key_file, path, G_KEY_FILE_NONE, NULL)) {
+	status = NOTMUCH_STATUS_FILE_ERROR;
+    }
+
+DONE:
+    talloc_free (local);
+    return status;
+}
+
 notmuch_status_t
 notmuch_database_open_with_config (const char *database_path,
 				   notmuch_database_mode_t mode,
@@ -49,7 +125,6 @@ notmuch_database_open_with_config (const char *database_path,
     void *local = talloc_new (NULL);
     notmuch_database_t *notmuch = NULL;
     char *notmuch_path, *xapian_path, *incompat_features;
-    char *configured_database_path = NULL;
     char *message = NULL;
     struct stat st;
     int err;
@@ -57,18 +132,14 @@ notmuch_database_open_with_config (const char *database_path,
     GKeyFile *key_file = NULL;
     static int initialized = 0;
 
-    /* XXX TODO: default locations for NULL case, handle profiles */
-    if (config_path != NULL && ! EMPTY_STRING (config_path)) {
-	key_file = g_key_file_new ();
-	if (! g_key_file_load_from_file (key_file, config_path, G_KEY_FILE_NONE, NULL)) {
-	    status = NOTMUCH_STATUS_FILE_ERROR;
-	    goto DONE;
-	}
-	configured_database_path = g_key_file_get_value (key_file, "database", "path", NULL);
+    status = _load_key_file (config_path, profile, &key_file);
+    if (status) {
+	message = strdup ("Error: cannot load config file");
+	goto DONE;
     }
-
-    if (database_path == NULL)
-	database_path = configured_database_path;
+	
+    if (! database_path && key_file)
+	database_path = g_key_file_get_value (key_file, "database", "path", NULL);
 
     if (database_path == NULL) {
 	message = strdup ("Error: Cannot open a database for a NULL path.\n");
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 5fb1bb87..2986284a 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -15,14 +15,21 @@ int main (int argc, char** argv)
    notmuch_database_t *db;
    char *val;
    notmuch_status_t stat;
+   char *msg = NULL;
 
-   EXPECT0(notmuch_database_open_with_config (argv[1],
+   for (int i=1; i<argc; i++)
+      if (strcmp (argv[i], "%NULL%") == 0) argv[i] = NULL;
+
+   stat = notmuch_database_open_with_config (argv[1],
                                               NOTMUCH_DATABASE_MODE_READ_WRITE,
                                               argv[2],
-                                              NULL,
+                                              argv[3],
                                               &db,
-                                              NULL));
-
+                                              &msg);
+   if (stat != NOTMUCH_STATUS_SUCCESS) {
+     fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : "");
+     exit (1);
+   }
 EOF
 
 cat <<EOF > c_tail
@@ -51,7 +58,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 
 test_begin_subtest "notmuch_database_get_config_list: empty list"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
    notmuch_config_list_t *list;
    EXPECT0(notmuch_database_get_config_list (db, "nonexistent", &list));
@@ -83,7 +90,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: all pairs"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
    notmuch_config_list_t *list;
    EXPECT0(notmuch_database_set_config (db, "zzzafter", "afterval"));
@@ -128,7 +135,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: one prefix"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
    notmuch_config_list_t *list;
    EXPECT0(notmuch_database_get_config_list (db, "test.key", &list));
@@ -147,7 +154,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "dump config"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
     EXPECT0(notmuch_database_set_config (db, "key with spaces", "value, with, spaces!"));
 }
@@ -165,7 +172,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "restore config"
 notmuch dump --include=config >EXPECTED
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
     EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
 }
@@ -194,4 +201,132 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+backup_database
+test_begin_subtest "override config from \${NOTMUCH_CONFIG}"
+notmuch config set test.key1 overridden
+# second argument omitted to make argv[2] == NULL
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+{
+   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
+   printf("test.key2 = %s\n", val);
+}
+EOF
+notmuch config set test.key1
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
+backup_database
+test_begin_subtest "override config from \${HOME}/.notmuch-config"
+ovconfig=${HOME}/.notmuch-config
+cp ${NOTMUCH_CONFIG} ${ovconfig}
+old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
+unset NOTMUCH_CONFIG
+notmuch --config=${ovconfig} config set test.key1 overridden-home
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL%
+{
+   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
+   printf("test.key2 = %s\n", val);
+}
+EOF
+rm -f ${ovconfig}
+NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden-home
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
+backup_database
+test_begin_subtest "override config from \${XDG_CONFIG_HOME}/notmuch"
+ovconfig=${HOME}/.config/notmuch/default/config
+mkdir -p $(dirname ${ovconfig})
+cp ${NOTMUCH_CONFIG} ${ovconfig}
+old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
+unset NOTMUCH_CONFIG
+notmuch --config=${ovconfig} config set test.key1 overridden-xdg
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL%
+{
+   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
+   printf("test.key2 = %s\n", val);
+}
+EOF
+rm -f ${ovconfig}
+NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden-xdg
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
+backup_database
+test_begin_subtest "override config from \${XDG_CONFIG_HOME}/notmuch with profile"
+ovconfig=${HOME}/.config/notmuch/work/config
+mkdir -p $(dirname ${ovconfig})
+cp ${NOTMUCH_CONFIG} ${ovconfig}
+old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
+unset NOTMUCH_CONFIG
+notmuch --config=${ovconfig} config set test.key1 overridden-xdg-profile
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% work
+{
+   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
+   printf("test.key2 = %s\n", val);
+}
+EOF
+rm -f ${ovconfig}
+NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden-xdg-profile
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
+backup_database
+test_begin_subtest "override config from \${HOME}/.notmuch-config.work (via args)"
+ovconfig=${HOME}/.notmuch-config.work
+cp ${NOTMUCH_CONFIG} ${ovconfig}
+old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
+unset NOTMUCH_CONFIG
+notmuch --config=${ovconfig} config set test.key1 overridden-profile
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% work
+{
+   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
+   printf("test.key2 = %s\n", val);
+}
+EOF
+#rm -f ${ovconfig}
+NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden-profile
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 test_done
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 05/38] CLI: generalize notmuch_config_mode_t
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (3 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 04/38] lib/open: add support for config profiles and default locations David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 06/38] lib/config: add notmuch_config_key_{get,set} David Bremner
                   ` (32 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The renaming and extra values will make sense when we start to convert
subcommands to the new configuration framework. It will also avoid
collisions with a new enum for configuration keys to be introduced in
a future commit.
---
 notmuch-client.h | 10 ++++++----
 notmuch-config.c |  6 +++---
 notmuch.c        | 36 ++++++++++++++++++------------------
 3 files changed, 27 insertions(+), 25 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index ebd43e8d..f59b3965 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -253,14 +253,16 @@ json_quote_str (const void *ctx, const char *str);
 /* notmuch-config.c */
 
 typedef enum {
-    NOTMUCH_CONFIG_OPEN		= 1 << 0,
-    NOTMUCH_CONFIG_CREATE	= 1 << 1,
-} notmuch_config_mode_t;
+    NOTMUCH_COMMAND_CONFIG_OPEN		= 1 << 0,
+    NOTMUCH_COMMAND_CONFIG_CREATE	= 1 << 1,
+    NOTMUCH_COMMAND_DATABASE_EARLY	= 1 << 2,
+    NOTMUCH_COMMAND_DATABASE_WRITE	= 1 << 3,
+} notmuch_command_mode_t;
 
 notmuch_config_t *
 notmuch_config_open (void *ctx,
 		     const char *filename,
-		     notmuch_config_mode_t config_mode);
+		     notmuch_command_mode_t config_mode);
 
 void
 notmuch_config_close (notmuch_config_t *config);
diff --git a/notmuch-config.c b/notmuch-config.c
index 19c2ddb3..cefb8274 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -324,7 +324,7 @@ get_config_from_file (notmuch_config_t *config, bool create_new)
 notmuch_config_t *
 notmuch_config_open (void *ctx,
 		     const char *filename,
-		     notmuch_config_mode_t config_mode)
+		     notmuch_command_mode_t config_mode)
 {
     GError *error = NULL;
     size_t tmp;
@@ -359,8 +359,8 @@ notmuch_config_open (void *ctx,
 
     config->key_file = g_key_file_new ();
 
-    if (config_mode & NOTMUCH_CONFIG_OPEN) {
-	bool create_new = (config_mode & NOTMUCH_CONFIG_CREATE) != 0;
+    if (config_mode & NOTMUCH_COMMAND_CONFIG_OPEN) {
+	bool create_new = (config_mode & NOTMUCH_COMMAND_CONFIG_CREATE) != 0;
 
 	if (! get_config_from_file (config, create_new)) {
 	    talloc_free (config);
diff --git a/notmuch.c b/notmuch.c
index 4ef1484f..314bf53e 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -33,7 +33,7 @@ typedef int (*command_function_t) (notmuch_config_t *config, int argc, char *arg
 typedef struct command {
     const char *name;
     command_function_t function;
-    notmuch_config_mode_t config_mode;
+    notmuch_command_mode_t mode;
     const char *summary;
 } command_t;
 
@@ -136,41 +136,41 @@ notmuch_process_shared_indexing_options (notmuch_database_t *notmuch)
 
 
 static command_t commands[] = {
-    { NULL, notmuch_command, NOTMUCH_CONFIG_OPEN | NOTMUCH_CONFIG_CREATE,
+    { NULL, notmuch_command, NOTMUCH_COMMAND_CONFIG_OPEN | NOTMUCH_COMMAND_CONFIG_CREATE,
       "Notmuch main command." },
-    { "setup", notmuch_setup_command, NOTMUCH_CONFIG_OPEN | NOTMUCH_CONFIG_CREATE,
+    { "setup", notmuch_setup_command, NOTMUCH_COMMAND_CONFIG_OPEN | NOTMUCH_COMMAND_CONFIG_CREATE,
       "Interactively set up notmuch for first use." },
-    { "new", notmuch_new_command, NOTMUCH_CONFIG_OPEN,
+    { "new", notmuch_new_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Find and import new messages to the notmuch database." },
-    { "insert", notmuch_insert_command, NOTMUCH_CONFIG_OPEN,
+    { "insert", notmuch_insert_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Add a new message into the maildir and notmuch database." },
-    { "search", notmuch_search_command, NOTMUCH_CONFIG_OPEN,
+    { "search", notmuch_search_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Search for messages matching the given search terms." },
-    { "address", notmuch_address_command, NOTMUCH_CONFIG_OPEN,
+    { "address", notmuch_address_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Get addresses from messages matching the given search terms." },
-    { "show", notmuch_show_command, NOTMUCH_CONFIG_OPEN,
+    { "show", notmuch_show_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Show all messages matching the search terms." },
-    { "count", notmuch_count_command, NOTMUCH_CONFIG_OPEN,
+    { "count", notmuch_count_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Count messages matching the search terms." },
-    { "reply", notmuch_reply_command, NOTMUCH_CONFIG_OPEN,
+    { "reply", notmuch_reply_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Construct a reply template for a set of messages." },
-    { "tag", notmuch_tag_command, NOTMUCH_CONFIG_OPEN,
+    { "tag", notmuch_tag_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Add/remove tags for all messages matching the search terms." },
-    { "dump", notmuch_dump_command, NOTMUCH_CONFIG_OPEN,
+    { "dump", notmuch_dump_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Create a plain-text dump of the tags for each message." },
-    { "restore", notmuch_restore_command, NOTMUCH_CONFIG_OPEN,
+    { "restore", notmuch_restore_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Restore the tags from the given dump file (see 'dump')." },
-    { "compact", notmuch_compact_command, NOTMUCH_CONFIG_OPEN,
+    { "compact", notmuch_compact_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Compact the notmuch database." },
-    { "reindex", notmuch_reindex_command, NOTMUCH_CONFIG_OPEN,
+    { "reindex", notmuch_reindex_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Re-index all messages matching the search terms." },
-    { "config", notmuch_config_command, NOTMUCH_CONFIG_OPEN,
+    { "config", notmuch_config_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Get or set settings in the notmuch configuration file." },
 #if WITH_EMACS
     { "emacs-mua", NULL, 0,
       "send mail with notmuch and emacs." },
 #endif
-    { "help", notmuch_help_command, NOTMUCH_CONFIG_CREATE, /* create but don't save config */
+    { "help", notmuch_help_command, NOTMUCH_COMMAND_CONFIG_CREATE, /* create but don't save config */
       "This message, or more detailed help for the named command." }
 };
 
@@ -496,7 +496,7 @@ main (int argc, char *argv[])
 	goto DONE;
     }
 
-    config = notmuch_config_open (local, config_file_name, command->config_mode);
+    config = notmuch_config_open (local, config_file_name, command->mode);
     if (! config) {
 	ret = EXIT_FAILURE;
 	goto DONE;
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 06/38] lib/config: add notmuch_config_key_{get,set}
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (4 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 05/38] CLI: generalize notmuch_config_mode_t David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 07/38] lib/open: load default values for known configuration keys David Bremner
                   ` (31 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

By using an enum we can have better error detection than copy pasting
key strings around.

The question of what layer this belongs in is a bit
tricky. Historically most of the keys are defined by the CLI. On the
other hand features like excludes are supported in the
library/bindings, and it makes sense to configure them from the
library as well.

The somewhat long prefix for notmuch_config_t is to avoid collisions
with the existing usage in notmuch-client.h.
---
 lib/config.cc          | 34 +++++++++++++++++++++++++
 lib/notmuch.h          | 56 ++++++++++++++++++++++++++++++++++++++++++
 test/T590-libconfig.sh | 38 ++++++++++++++++++++++++++++
 3 files changed, 128 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 8bce7ba8..c92ffeef 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -279,3 +279,37 @@ _notmuch_config_load_from_file (notmuch_database_t *notmuch,
  DONE:
     return status;
 }
+
+const char *
+_notmuch_config_key_to_string (notmuch_config_key_t key) {
+    switch (key) {
+    case NOTMUCH_CONFIG_DATABASE_PATH:
+	return "database.path";
+    case NOTMUCH_CONFIG_EXCLUDE_TAGS:
+	return "search.exclude_tags";
+    case NOTMUCH_CONFIG_NEW_TAGS:
+	return "new.tags";
+    case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS:
+	return "maildir.synchronize_flags";
+    case NOTMUCH_CONFIG_PRIMARY_EMAIL:
+	return "user.primary_email";
+    case NOTMUCH_CONFIG_OTHER_EMAIL:
+	return "user.other_email";
+    case NOTMUCH_CONFIG_USER_NAME:
+	return "user.name";
+    default:
+	return NULL;
+    }
+}
+
+const char *
+notmuch_config_get (notmuch_database_t *notmuch, notmuch_config_key_t key) {
+
+    return _notmuch_string_map_get (notmuch->config, _notmuch_config_key_to_string (key));
+}
+
+notmuch_status_t
+notmuch_config_set (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val) {
+
+    return notmuch_database_set_config (notmuch, _notmuch_config_key_to_string (key), val);
+}
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 9346640a..5c35b44b 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2325,6 +2325,11 @@ notmuch_filenames_destroy (notmuch_filenames_t *filenames);
  * set config 'key' to 'value'
  *
  * @since libnotmuch 4.4 (notmuch 0.23)
+ * @retval #NOTMUCH_STATUS_READ_ONLY_DATABASE: Database was opened in
+ *	read-only mode so message cannot be modified.
+ * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION: an exception was thrown
+ *      accessing the database.
+ * @retval #NOTMUCH_STATUS_SUCCESS
  */
 notmuch_status_t
 notmuch_database_set_config (notmuch_database_t *db, const char *key, const char *value);
@@ -2339,6 +2344,7 @@ notmuch_database_set_config (notmuch_database_t *db, const char *key, const char
  * caller.
  *
  * @since libnotmuch 4.4 (notmuch 0.23)
+ *
  */
 notmuch_status_t
 notmuch_database_get_config (notmuch_database_t *db, const char *key, char **value);
@@ -2400,6 +2406,56 @@ void
 notmuch_config_list_destroy (notmuch_config_list_t *config_list);
 
 
+/**
+ * Configuration keys known to libnotmuch
+ */
+typedef enum _notmuch_config_key {
+    NOTMUCH_CONFIG_FIRST,
+    NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST,
+    NOTMUCH_CONFIG_EXCLUDE_TAGS,
+    NOTMUCH_CONFIG_NEW_TAGS,
+    NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS,
+    NOTMUCH_CONFIG_PRIMARY_EMAIL,
+    NOTMUCH_CONFIG_OTHER_EMAIL,
+    NOTMUCH_CONFIG_USER_NAME,
+    NOTMUCH_CONFIG_LAST
+} notmuch_config_key_t;
+
+/**
+ * get a configuration value from an open database.
+ *
+ * This value reflects all configuration information given at the time
+ * the database was opened.
+ *
+ * @param[in] notmuch database
+ * @param[in] key configuration key
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ * @retval NULL if 'key' unknown or if no value is known for
+ *         'key'.  Otherwise returns a string owned by notmuch which should
+ *         not be modified nor freed by the caller.
+ */
+const char *
+notmuch_config_get (notmuch_database_t *notmuch, notmuch_config_key_t key);
+
+/**
+ * set a configuration value from in an open database.
+ *
+ * This value reflects all configuration information given at the time
+ * the database was opened.
+ *
+ * @param[in,out] notmuch database open read/write
+ * @param[in] key configuration key
+ * @param[in] val configuration value
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ * @retval returns any return value for notmuch_database_set_config.
+ */
+notmuch_status_t
+notmuch_config_set (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val);
+
 /**
  * get the current default indexing options for a given database.
  *
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 2986284a..8bf43eed 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -201,6 +201,44 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+backup_database
+test_begin_subtest "get config by key"
+notmuch config set test.key1 overridden
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+{
+   printf("before = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS));
+   EXPECT0(notmuch_database_set_config (db, "maildir.synchronize_flags", "false"));
+   printf("after = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS));
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+before = true
+after = false
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
+backup_database
+test_begin_subtest "set config by key"
+notmuch config set test.key1 overridden
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+{
+   printf("before = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS));
+   EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, "false"));
+   printf("after = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS));
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+before = true
+after = false
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 backup_database
 test_begin_subtest "override config from \${NOTMUCH_CONFIG}"
 notmuch config set test.key1 overridden
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 07/38] lib/open: load default values for known configuration keys.
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (5 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 06/38] lib/config: add notmuch_config_key_{get,set} David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 08/38] CLI: add (unused) database argument to subcommands David Bremner
                   ` (30 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This emulates the behaviour of notmuch_config_open defined in the CLI,
in that it fills in default values if they are not otherwise defined.
---
 lib/config.cc          | 50 +++++++++++++++++++++++++++++++++++++++++-
 lib/notmuch-private.h  |  3 +++
 lib/open.cc            |  4 ++++
 test/T590-libconfig.sh | 27 +++++++++++++++++++++++
 4 files changed, 83 insertions(+), 1 deletion(-)

diff --git a/lib/config.cc b/lib/config.cc
index c92ffeef..f6d9bf44 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -31,6 +31,8 @@ struct _notmuch_config_list {
     char *current_val;
 };
 
+static const char * _notmuch_config_key_to_string (notmuch_config_key_t key);
+
 static int
 _notmuch_config_list_destroy (notmuch_config_list_t *list)
 {
@@ -280,7 +282,7 @@ _notmuch_config_load_from_file (notmuch_database_t *notmuch,
     return status;
 }
 
-const char *
+static const char *
 _notmuch_config_key_to_string (notmuch_config_key_t key) {
     switch (key) {
     case NOTMUCH_CONFIG_DATABASE_PATH:
@@ -302,6 +304,52 @@ _notmuch_config_key_to_string (notmuch_config_key_t key) {
     }
 }
 
+static const char *
+_notmuch_config_default (void *ctx, notmuch_config_key_t key) {
+    char *path;
+
+    switch (key) {
+    case NOTMUCH_CONFIG_DATABASE_PATH:
+	path = getenv ("MAILDIR");
+	if (path)
+	    path = talloc_strdup (ctx, path);
+	else
+	    path = talloc_asprintf (ctx, "%s/mail",
+				    getenv ("HOME"));
+	return path;
+    case NOTMUCH_CONFIG_EXCLUDE_TAGS:
+	return "";
+    case NOTMUCH_CONFIG_NEW_TAGS:
+	return "inbox;unread";
+    case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS:
+	return "true";
+    case NOTMUCH_CONFIG_USER_NAME:
+    case NOTMUCH_CONFIG_PRIMARY_EMAIL:
+    case NOTMUCH_CONFIG_OTHER_EMAIL:
+	return NULL;
+    default:
+    case NOTMUCH_CONFIG_LAST:
+	INTERNAL_ERROR ("illegal key enum %d", key);
+   }
+}
+
+notmuch_status_t
+_notmuch_config_load_defaults (notmuch_database_t *notmuch) {
+    notmuch_config_key_t key;
+    for (key = NOTMUCH_CONFIG_FIRST;
+	 key < NOTMUCH_CONFIG_LAST;
+	 key = notmuch_config_key_t(key + 1)) {
+	const char *val = notmuch_config_get (notmuch, key);
+	const char *key_string = _notmuch_config_key_to_string (key);
+
+	val = _notmuch_string_map_get (notmuch->config, key_string);
+	if (! val) {
+	    _notmuch_string_map_set (notmuch->config, key_string, _notmuch_config_default (notmuch, key));
+	}
+    }
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 const char *
 notmuch_config_get (notmuch_database_t *notmuch, notmuch_config_key_t key) {
 
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 40b1a855..961d50cf 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -710,6 +710,9 @@ _notmuch_config_load_from_database (notmuch_database_t * db);
 
 notmuch_status_t
 _notmuch_config_load_from_file (notmuch_database_t * db, GKeyFile *file);
+
+notmuch_status_t
+_notmuch_config_load_defaults (notmuch_database_t * db);
 NOTMUCH_END_DECLS
 
 #ifdef __cplusplus
diff --git a/lib/open.cc b/lib/open.cc
index 76255283..8cede121 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -288,6 +288,10 @@ notmuch_database_open_with_config (const char *database_path,
 	if (status)
 	    goto DONE;
 
+	status = _notmuch_config_load_defaults (notmuch);
+	if (status)
+	    goto DONE;
+
 	status = _notmuch_database_setup_standard_query_fields (notmuch);
 	if (status)
 	    goto DONE;
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 8bf43eed..0c7398a2 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -239,6 +239,33 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+test_begin_subtest "load default values"
+export MAILDIR=${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} '' %NULL%
+{
+    notmuch_config_key_t key;
+    for (key = NOTMUCH_CONFIG_FIRST;
+	 key < NOTMUCH_CONFIG_LAST;
+	 key = (notmuch_config_key_t)(key + 1)) {
+	const char *val = notmuch_config_get (db, key);
+        printf("%s\n", val ? val : "NULL" );
+    }
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+MAIL_DIR
+
+inbox;unread
+true
+NULL
+NULL
+NULL
+== stderr ==
+EOF
+unset MAILDIR
+test_expect_equal_file EXPECTED OUTPUT
+
 backup_database
 test_begin_subtest "override config from \${NOTMUCH_CONFIG}"
 notmuch config set test.key1 overridden
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 08/38] CLI: add (unused) database argument to subcommands.
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (6 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 07/38] lib/open: load default values for known configuration keys David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 09/38] util: add strsplit_len: simplified strtok with delimiter escaping David Bremner
                   ` (29 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will allow transitioning individual subcommands to the new
configuration framework. Eventually when they are all converted we can
remove the notmuch_config_t * argument.

For now, live with the parameter shadowing in some some subcommands;
it will go away when they are converted.
---
 notmuch-client.h  | 28 ++++++++++++++--------------
 notmuch-compact.c |  2 +-
 notmuch-config.c  |  2 +-
 notmuch-count.c   |  2 +-
 notmuch-dump.c    |  2 +-
 notmuch-insert.c  |  2 +-
 notmuch-new.c     |  2 +-
 notmuch-reindex.c |  2 +-
 notmuch-reply.c   |  2 +-
 notmuch-restore.c |  2 +-
 notmuch-search.c  |  4 ++--
 notmuch-setup.c   |  1 +
 notmuch-show.c    |  2 +-
 notmuch-tag.c     |  2 +-
 notmuch.c         | 16 ++++++++++------
 15 files changed, 38 insertions(+), 33 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index f59b3965..e8fb0323 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -171,46 +171,46 @@ void
 notmuch_exit_if_unsupported_format (void);
 
 int
-notmuch_count_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_count_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_dump_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_new_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_insert_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_reindex_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_reply_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_restore_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_search_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_search_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_address_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_address_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_setup_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_setup_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_show_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_tag_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_config_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_config_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 int
-notmuch_compact_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_compact_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 const char *
 notmuch_time_relative_date (const void *ctx, time_t then);
diff --git a/notmuch-compact.c b/notmuch-compact.c
index f8996cf4..ab2066e1 100644
--- a/notmuch-compact.c
+++ b/notmuch-compact.c
@@ -27,7 +27,7 @@ status_update_cb (const char *msg, unused (void *closure))
 }
 
 int
-notmuch_compact_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_compact_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     const char *path = notmuch_config_get_database_path (config);
     const char *backup_path = NULL;
diff --git a/notmuch-config.c b/notmuch-config.c
index cefb8274..4fa274c7 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -1102,7 +1102,7 @@ notmuch_config_command_list (notmuch_config_t *config)
 }
 
 int
-notmuch_config_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_config_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     int ret;
     int opt_index;
diff --git a/notmuch-count.c b/notmuch-count.c
index d8ad7d6d..f752ef62 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -148,7 +148,7 @@ count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags,
 }
 
 int
-notmuch_count_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_count_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_database_t *notmuch;
     char *query_str;
diff --git a/notmuch-dump.c b/notmuch-dump.c
index 887ef7f0..eb629dc9 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -361,7 +361,7 @@ notmuch_database_dump (notmuch_database_t *notmuch,
 }
 
 int
-notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_dump_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_database_t *notmuch;
     const char *query_str = NULL;
diff --git a/notmuch-insert.c b/notmuch-insert.c
index 1d3b0150..de160309 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -444,7 +444,7 @@ add_file (notmuch_database_t *notmuch, const char *path, tag_op_list_t *tag_ops,
 }
 
 int
-notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_status_t status, close_status;
     notmuch_database_t *notmuch;
diff --git a/notmuch-new.c b/notmuch-new.c
index 4075d395..e0e3de25 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -1043,7 +1043,7 @@ print_results (const add_files_state_t *state)
 }
 
 int
-notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_database_t *notmuch;
     add_files_state_t add_files_state = {
diff --git a/notmuch-reindex.c b/notmuch-reindex.c
index 5a39ade1..9d337c48 100644
--- a/notmuch-reindex.c
+++ b/notmuch-reindex.c
@@ -83,7 +83,7 @@ reindex_query (notmuch_database_t *notmuch, const char *query_string,
 }
 
 int
-notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_reindex_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     char *query_string = NULL;
     notmuch_database_t *notmuch;
diff --git a/notmuch-reply.c b/notmuch-reply.c
index ceb4f39b..a8ffbf75 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -700,7 +700,7 @@ do_reply (notmuch_config_t *config,
 }
 
 int
-notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_reply_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_database_t *notmuch;
     notmuch_query_t *query;
diff --git a/notmuch-restore.c b/notmuch-restore.c
index e2dc3d45..544f4228 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -219,7 +219,7 @@ parse_sup_line (void *ctx, char *line,
 }
 
 int
-notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_restore_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_database_t *notmuch;
     bool accumulate = false;
diff --git a/notmuch-search.c b/notmuch-search.c
index 2805d960..34e27058 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -805,7 +805,7 @@ static const notmuch_opt_desc_t common_options[] = {
 };
 
 int
-notmuch_search_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_search_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     search_context_t *ctx = &search_context;
     int opt_index, ret;
@@ -871,7 +871,7 @@ notmuch_search_command (notmuch_config_t *config, int argc, char *argv[])
 }
 
 int
-notmuch_address_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_address_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     search_context_t *ctx = &search_context;
     int opt_index, ret;
diff --git a/notmuch-setup.c b/notmuch-setup.c
index cd1a52ff..67214470 100644
--- a/notmuch-setup.c
+++ b/notmuch-setup.c
@@ -122,6 +122,7 @@ parse_tag_list (void *ctx, char *response)
 
 int
 notmuch_setup_command (notmuch_config_t *config,
+		       unused(notmuch_database_t *notmuch),
 		       int argc, char *argv[])
 {
     char *response = NULL;
diff --git a/notmuch-show.c b/notmuch-show.c
index dd836add..04b90cd7 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1215,7 +1215,7 @@ static const notmuch_show_format_t *formatters[] = {
 };
 
 int
-notmuch_show_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_show_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     notmuch_database_t *notmuch;
     notmuch_query_t *query;
diff --git a/notmuch-tag.c b/notmuch-tag.c
index 05b1837d..205f2733 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -187,7 +187,7 @@ tag_file (void *ctx, notmuch_database_t *notmuch, tag_op_flag_t flags,
 }
 
 int
-notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[])
+notmuch_tag_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     tag_op_list_t *tag_ops = NULL;
     char *query_string = NULL;
diff --git a/notmuch.c b/notmuch.c
index 314bf53e..fd4a7945 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -27,8 +27,11 @@
  *
  * The return value will be used as notmuch exit status code,
  * preferably EXIT_SUCCESS or EXIT_FAILURE.
+ *
+ * Each subcommand should be passed either a config object, or an open
+ * database
  */
-typedef int (*command_function_t) (notmuch_config_t *config, int argc, char *argv[]);
+typedef int (*command_function_t) (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 typedef struct command {
     const char *name;
@@ -38,10 +41,10 @@ typedef struct command {
 } command_t;
 
 static int
-notmuch_help_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_help_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 static int
-notmuch_command (notmuch_config_t *config, int argc, char *argv[]);
+notmuch_command (notmuch_config_t *config, notmuch_database_t *notmuch, int argc, char *argv[]);
 
 static int
 _help_for (const char *topic);
@@ -335,7 +338,7 @@ _help_for (const char *topic_name)
 }
 
 static int
-notmuch_help_command (unused (notmuch_config_t *config), int argc, char *argv[])
+notmuch_help_command (unused (notmuch_config_t *config), unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
     int opt_index;
 
@@ -360,6 +363,7 @@ notmuch_help_command (unused (notmuch_config_t *config), int argc, char *argv[])
  */
 static int
 notmuch_command (notmuch_config_t *config,
+		 unused(notmuch_database_t *notmuch),
 		 unused(int argc), unused(char **argv))
 {
     char *db_path;
@@ -369,7 +373,7 @@ notmuch_command (notmuch_config_t *config,
      * notmuch_setup_command which will give a nice welcome message,
      * and interactively guide the user through the configuration. */
     if (notmuch_config_is_new (config))
-	return notmuch_setup_command (config, 0, NULL);
+	return notmuch_setup_command (config, NULL, 0, NULL);
 
     /* Notmuch is already configured, but is there a database? */
     db_path = talloc_asprintf (config, "%s/%s",
@@ -502,7 +506,7 @@ main (int argc, char *argv[])
 	goto DONE;
     }
 
-    ret = (command->function)(config, argc - opt_index, argv + opt_index);
+    ret = (command->function)(config, NULL, argc - opt_index, argv + opt_index);
 
   DONE:
     if (config)
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 09/38] util: add strsplit_len: simplified strtok with delimiter escaping
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (7 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 08/38] CLI: add (unused) database argument to subcommands David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 10/38] lib/config: add config values iterator David Bremner
                   ` (28 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will be used to make iterators for configuration values.
---
 util/string-util.c | 23 +++++++++++++++++++++++
 util/string-util.h | 14 ++++++++++++++
 2 files changed, 37 insertions(+)

diff --git a/util/string-util.c b/util/string-util.c
index de8430b2..27f8a26b 100644
--- a/util/string-util.c
+++ b/util/string-util.c
@@ -24,6 +24,7 @@
 
 #include <ctype.h>
 #include <errno.h>
+#include <stdbool.h>
 
 char *
 strtok_len (char *s, const char *delim, size_t *len)
@@ -37,6 +38,28 @@ strtok_len (char *s, const char *delim, size_t *len)
     return *len ? s : NULL;
 }
 
+const char *
+strsplit_len (const char *s, char delim, size_t *len)
+{
+    bool escaping = false;
+    size_t count = 0;
+
+    /* Skip initial unescaped delimiters */
+    while (*s && *s == delim)
+	s++;
+
+    while (s[count] && (escaping || s[count] != delim)) {
+	escaping = (s[count] == '\\');
+	count++;
+    }
+
+    if (count==0)
+	return NULL;
+
+    *len = count;
+    return s;
+}
+
 const char *
 strtok_len_c (const char *s, const char *delim, size_t *len)
 {
diff --git a/util/string-util.h b/util/string-util.h
index fb95a740..80647c5f 100644
--- a/util/string-util.h
+++ b/util/string-util.h
@@ -26,6 +26,20 @@ char *strtok_len (char *s, const char *delim, size_t *len);
 /* Const version of strtok_len. */
 const char *strtok_len_c (const char *s, const char *delim, size_t *len);
 
+/* Simplified version of strtok_len, with a single delimiter.
+ * Handles escaping delimiters with \
+ * Usage pattern:
+ *
+ * const char *tok = input;
+ * const char *delim = ';';
+ * size_t tok_len = 0;
+ *
+ * while ((tok = strsplit_len (tok + tok_len, delim, &tok_len)) != NULL) {
+ *     // do stuff with string tok of length tok_len
+ * }
+ */
+const char *strsplit_len (const char *s, char delim, size_t *len);
+
 /* Return a talloced string with str sanitized.
  *
  * Whitespace characters (tabs and newlines) are replaced with spaces,
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 10/38] lib/config: add config values iterator
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (8 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 09/38] util: add strsplit_len: simplified strtok with delimiter escaping David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 11/38] lib/config: make values iterators restartable David Bremner
                   ` (27 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is intended to avoid duplicating the string splitting and
traversal code for all clients of the config API.
---
 lib/config.cc          | 52 ++++++++++++++++++++++++++++++++++
 lib/notmuch.h          | 64 ++++++++++++++++++++++++++++++++++++++++++
 notmuch.c              | 35 +++++++++++++++++++----
 test/T590-libconfig.sh | 48 ++++++++++++++++++++++++++++++-
 4 files changed, 192 insertions(+), 7 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index f6d9bf44..4500fe1a 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -31,6 +31,11 @@ struct _notmuch_config_list {
     char *current_val;
 };
 
+struct _notmuch_config_values {
+    const char *iterator;
+    size_t tok_len;
+};
+
 static const char * _notmuch_config_key_to_string (notmuch_config_key_t key);
 
 static int
@@ -248,6 +253,53 @@ _notmuch_config_load_from_database (notmuch_database_t *notmuch)
     return status;
 }
 
+notmuch_config_values_t *
+notmuch_config_get_values (notmuch_database_t *notmuch, notmuch_config_key_t key)
+{
+    notmuch_config_values_t *values;
+
+    const char *str;
+    const char *key_str = _notmuch_config_key_to_string (key);
+
+    if (! key_str)
+	return NULL;
+
+    str  = _notmuch_string_map_get (notmuch->config, key_str);
+    if (! str)
+	return NULL;
+
+    values = talloc (notmuch, notmuch_config_values_t);
+    if (unlikely(! values))
+	return NULL;
+
+    values->iterator = strsplit_len (str, ';', &(values->tok_len));
+    return values;
+}
+
+notmuch_bool_t
+notmuch_config_values_valid (notmuch_config_values_t *values) {
+    if (! values)
+	return false;
+
+    return (values->iterator != NULL);
+}
+
+const char *
+notmuch_config_values_get (notmuch_config_values_t *values) {
+    return talloc_strndup (values, values->iterator, values->tok_len);
+}
+
+void
+notmuch_config_values_move_to_next (notmuch_config_values_t *values) {
+    values->iterator += values->tok_len;
+    values->iterator = strsplit_len (values->iterator, ';', &(values->tok_len));
+}
+
+void
+notmuch_config_values_destroy (notmuch_config_values_t *values) {
+    talloc_free (values);
+}
+
 notmuch_status_t
 _notmuch_config_load_from_file (notmuch_database_t *notmuch,
 				GKeyFile *file)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 5c35b44b..7aeff567 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -236,6 +236,7 @@ typedef struct _notmuch_tags notmuch_tags_t;
 typedef struct _notmuch_directory notmuch_directory_t;
 typedef struct _notmuch_filenames notmuch_filenames_t;
 typedef struct _notmuch_config_list notmuch_config_list_t;
+typedef struct _notmuch_config_values notmuch_config_values_t;
 typedef struct _notmuch_indexopts notmuch_indexopts_t;
 #endif /* __DOXYGEN__ */
 
@@ -2456,6 +2457,69 @@ notmuch_config_get (notmuch_database_t *notmuch, notmuch_config_key_t key);
 notmuch_status_t
 notmuch_config_set (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val);
 
+/**
+ * Returns an iterator for a ';'-delimited list of configuration values
+ *
+ * These values reflect all configuration information given at the
+ * time the database was opened.
+ *
+ * @param[in] notmuch database
+ * @param[in] key configuration key
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ * @retval NULL in case of error.
+ */
+notmuch_config_values_t *
+notmuch_config_get_values (notmuch_database_t *notmuch, notmuch_config_key_t key);
+
+/**
+ * Is the given 'config_values' iterator pointing at a valid element.
+ *
+ * @param[in] values iterator
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ * @retval FALSE if passed a NULL pointer, or the iterator is exhausted.
+ *
+ */
+notmuch_bool_t
+notmuch_config_values_valid (notmuch_config_values_t *values);
+
+/**
+ * Get the current value from the 'values' iterator
+ *
+ * @param[in] values iterator
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ * @retval a string with the same lifetime as the iterator
+ */
+const char *
+notmuch_config_values_get (notmuch_config_values_t *values);
+
+/**
+ * Move the 'values' iterator to the next element
+ *
+ * @param[in,out] values iterator
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ */
+void
+notmuch_config_values_move_to_next (notmuch_config_values_t *values);
+
+/**
+ * Destroy a config values iterator, along with any associated
+ * resources.
+ *
+ * @param[in,out] values iterator
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ */
+void
+notmuch_config_values_destroy (notmuch_config_values_t *values);
+
 /**
  * get the current default indexing options for a given database.
  *
diff --git a/notmuch.c b/notmuch.c
index fd4a7945..e0649048 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -457,6 +457,7 @@ main (int argc, char *argv[])
     command_t *command;
     const char *config_file_name = NULL;
     notmuch_config_t *config = NULL;
+    notmuch_database_t *notmuch = NULL;
     int opt_index;
     int ret;
 
@@ -500,13 +501,35 @@ main (int argc, char *argv[])
 	goto DONE;
     }
 
-    config = notmuch_config_open (local, config_file_name, command->mode);
-    if (! config) {
-	ret = EXIT_FAILURE;
-	goto DONE;
-    }
+    if (command->mode & NOTMUCH_COMMAND_DATABASE_EARLY) {
+	char *status_string = NULL;
+	notmuch_database_mode_t mode;
+	if (command->mode & NOTMUCH_COMMAND_DATABASE_WRITE)
+	    mode = NOTMUCH_DATABASE_MODE_READ_WRITE;
+	else
+	    mode = NOTMUCH_DATABASE_MODE_READ_ONLY;
+
+	if (notmuch_database_open_with_config (NULL,
+					       mode,
+					       config_file_name,
+					       NULL,
+					       &notmuch,
+					       &status_string)) {
+	    if (status_string) {
+		fputs (status_string, stderr);
+		free (status_string);
+	    }
 
-    ret = (command->function)(config, NULL, argc - opt_index, argv + opt_index);
+	    return EXIT_FAILURE;
+	}
+    } else {
+	config = notmuch_config_open (local, config_file_name, command->mode);
+	if (! config) {
+	    ret = EXIT_FAILURE;
+	    goto DONE;
+	}
+    }
+    ret = (command->function)(config, notmuch, argc - opt_index, argv + opt_index);
 
   DONE:
     if (config)
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 0c7398a2..6eaec101 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -201,9 +201,55 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+test_begin_subtest "notmuch_config_get_values"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+    notmuch_config_values_t *values;
+    EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, "a;b;c"));
+    for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS);
+	 notmuch_config_values_valid (values);
+	 notmuch_config_values_move_to_next (values))
+    {
+	  puts (notmuch_config_values_get (values));
+    }
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+a
+b
+c
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
+backup_database
+test_begin_subtest "notmuch_config_get_values, trailing ;"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+    notmuch_config_values_t *values;
+    EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, "a;b;c"));
+    for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS);
+	 notmuch_config_values_valid (values);
+	 notmuch_config_values_move_to_next (values))
+    {
+	  puts (notmuch_config_values_get (values));
+    }
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+a
+b
+c
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 backup_database
 test_begin_subtest "get config by key"
-notmuch config set test.key1 overridden
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
    printf("before = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS));
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 11/38] lib/config: make values iterators restartable
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (9 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 10/38] lib/config: add config values iterator David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 12/38] CLI/count: switch to new configuration framework David Bremner
                   ` (26 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is relatively cheap, and makes it easier to transform existing
code which uses arrays of pointers to store configuration lists.
---
 lib/config.cc          | 43 +++++++++++++++++++++++++++++++++---------
 lib/notmuch.h          | 12 ++++++++++++
 test/T590-libconfig.sh | 32 +++++++++++++++++++++++++++++++
 3 files changed, 78 insertions(+), 9 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index 4500fe1a..51e359b8 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -34,6 +34,8 @@ struct _notmuch_config_list {
 struct _notmuch_config_values {
     const char *iterator;
     size_t tok_len;
+    const char *string;
+    void *children; /* talloc_context */
 };
 
 static const char * _notmuch_config_key_to_string (notmuch_config_key_t key);
@@ -256,23 +258,33 @@ _notmuch_config_load_from_database (notmuch_database_t *notmuch)
 notmuch_config_values_t *
 notmuch_config_get_values (notmuch_database_t *notmuch, notmuch_config_key_t key)
 {
-    notmuch_config_values_t *values;
+    notmuch_config_values_t *values = NULL;
+    bool ok = false;
 
-    const char *str;
     const char *key_str = _notmuch_config_key_to_string (key);
 
     if (! key_str)
-	return NULL;
-
-    str  = _notmuch_string_map_get (notmuch->config, key_str);
-    if (! str)
-	return NULL;
+	goto DONE;
 
     values = talloc (notmuch, notmuch_config_values_t);
     if (unlikely(! values))
-	return NULL;
+	goto DONE;
+
+    values->children = talloc_new (values);
+
+    values->string = _notmuch_string_map_get (notmuch->config, key_str);
+    if (! values->string)
+	goto DONE;
+
+    values->iterator = strsplit_len (values->string, ';', &(values->tok_len));
+    ok = true;
 
-    values->iterator = strsplit_len (str, ';', &(values->tok_len));
+ DONE:
+    if (!ok) {
+	if (values)
+	    talloc_free(values);
+	return NULL;
+    }
     return values;
 }
 
@@ -289,6 +301,19 @@ notmuch_config_values_get (notmuch_config_values_t *values) {
     return talloc_strndup (values, values->iterator, values->tok_len);
 }
 
+void
+notmuch_config_values_start (notmuch_config_values_t *values) {
+    if (values == NULL)
+	return;
+    if (values->children) {
+	talloc_free (values->children);
+    }
+
+    values->children = talloc_new (values);
+
+    values->iterator = strsplit_len (values->string, ';', &(values->tok_len));
+}
+
 void
 notmuch_config_values_move_to_next (notmuch_config_values_t *values) {
     values->iterator += values->tok_len;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 7aeff567..39f39423 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2509,6 +2509,18 @@ notmuch_config_values_get (notmuch_config_values_t *values);
 void
 notmuch_config_values_move_to_next (notmuch_config_values_t *values);
 
+
+/**
+ * reset the 'values' iterator to the first element
+ *
+ * @param[in,out] values iterator. A NULL value is ignored.
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ */
+void
+notmuch_config_values_start (notmuch_config_values_t *values);
+
 /**
  * Destroy a config values iterator, along with any associated
  * resources.
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 6eaec101..5ffe967b 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -224,6 +224,38 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+test_begin_subtest "notmuch_config_get_values (restart)"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+    notmuch_config_values_t *values;
+    EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, "a;b;c"));
+    for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS);
+	 notmuch_config_values_valid (values);
+	 notmuch_config_values_move_to_next (values))
+    {
+	  puts (notmuch_config_values_get (values));
+    }
+    for (notmuch_config_values_start (values);
+	 notmuch_config_values_valid (values);
+	 notmuch_config_values_move_to_next (values))
+    {
+	  puts (notmuch_config_values_get (values));
+    }
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+a
+b
+c
+a
+b
+c
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 backup_database
 test_begin_subtest "notmuch_config_get_values, trailing ;"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 12/38] CLI/count: switch to new configuration framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (10 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 11/38] lib/config: make values iterators restartable David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 13/38] cli/dump: convert to new config framework David Bremner
                   ` (25 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The main effort is changing from the old argv style config list
iterators to the new more opaque ones provided by the library (and
backed by the database+file config cache).
---
 notmuch-count.c       | 48 +++++++++++++++++++------------------------
 notmuch.c             |  2 +-
 test/T060-count.sh    | 11 ++++++++++
 test/T140-excludes.sh | 15 ++++++++++++++
 4 files changed, 48 insertions(+), 28 deletions(-)

diff --git a/notmuch-count.c b/notmuch-count.c
index f752ef62..048b1f44 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -64,10 +64,9 @@ count_files (notmuch_query_t *query)
 /* return 0 on success, -1 on failure */
 static int
 print_count (notmuch_database_t *notmuch, const char *query_str,
-	     const char **exclude_tags, size_t exclude_tags_length, int output, int print_lastmod)
+	     notmuch_config_values_t *exclude_tags, int output, int print_lastmod)
 {
     notmuch_query_t *query;
-    size_t i;
     int count;
     unsigned int ucount;
     unsigned long revision;
@@ -81,13 +80,18 @@ print_count (notmuch_database_t *notmuch, const char *query_str,
 	return -1;
     }
 
-    for (i = 0; i < exclude_tags_length; i++) {
-	status = notmuch_query_add_tag_exclude (query, exclude_tags[i]);
-	if (status && status != NOTMUCH_STATUS_IGNORED) {
-	    print_status_query ("notmuch count", query, status);
-	    return -1;
+    for (notmuch_config_values_start (exclude_tags);
+	 notmuch_config_values_valid (exclude_tags);
+	 notmuch_config_values_move_to_next (exclude_tags)) {
+
+	status = notmuch_query_add_tag_exclude (query,
+						notmuch_config_values_get (exclude_tags));
+	    if (status && status != NOTMUCH_STATUS_IGNORED) {
+		print_status_query ("notmuch count", query, status);
+		ret = -1;
+		goto DONE;
+	    }
 	}
-    }
 
     switch (output) {
     case OUTPUT_MESSAGES:
@@ -127,8 +131,8 @@ print_count (notmuch_database_t *notmuch, const char *query_str,
 }
 
 static int
-count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags,
-	    size_t exclude_tags_length, int output, int print_lastmod)
+count_file (notmuch_database_t *notmuch, FILE *input, notmuch_config_values_t *exclude_tags,
+	    int output, int print_lastmod)
 {
     char *line = NULL;
     ssize_t line_len;
@@ -137,8 +141,7 @@ count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags,
 
     while (! ret && (line_len = getline (&line, &line_size, input)) != -1) {
 	chomp_newline (line);
-	ret = print_count (notmuch, line, exclude_tags, exclude_tags_length,
-			   output, print_lastmod);
+	ret = print_count (notmuch, line, exclude_tags, output, print_lastmod);
     }
 
     if (line)
@@ -148,15 +151,13 @@ count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags,
 }
 
 int
-notmuch_count_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_count_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
-    notmuch_database_t *notmuch;
     char *query_str;
     int opt_index;
     int output = OUTPUT_MESSAGES;
     bool exclude = true;
-    const char **search_exclude_tags = NULL;
-    size_t search_exclude_tags_length = 0;
+    notmuch_config_values_t *exclude_tags = NULL;
     bool batch = false;
     bool print_lastmod = false;
     FILE *input = stdin;
@@ -200,29 +201,22 @@ notmuch_count_command (notmuch_config_t *config, unused(notmuch_database_t *notm
 	return EXIT_FAILURE;
     }
 
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_ONLY, &notmuch))
-	return EXIT_FAILURE;
-
     notmuch_exit_if_unmatched_db_uuid (notmuch);
 
-    query_str = query_string_from_args (config, argc - opt_index, argv + opt_index);
+    query_str = query_string_from_args (notmuch, argc - opt_index, argv + opt_index);
     if (query_str == NULL) {
 	fprintf (stderr, "Out of memory.\n");
 	return EXIT_FAILURE;
     }
 
     if (exclude) {
-	search_exclude_tags = notmuch_config_get_search_exclude_tags
-				  (config, &search_exclude_tags_length);
+	exclude_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS);
     }
 
     if (batch)
-	ret = count_file (notmuch, input, search_exclude_tags,
-			  search_exclude_tags_length, output, print_lastmod);
+	ret = count_file (notmuch, input, exclude_tags, output, print_lastmod);
     else
-	ret = print_count (notmuch, query_str, search_exclude_tags,
-			   search_exclude_tags_length, output, print_lastmod);
+	ret = print_count (notmuch, query_str, exclude_tags, output, print_lastmod);
 
     notmuch_database_destroy (notmuch);
 
diff --git a/notmuch.c b/notmuch.c
index e0649048..40527893 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -153,7 +153,7 @@ static command_t commands[] = {
       "Get addresses from messages matching the given search terms." },
     { "show", notmuch_show_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Show all messages matching the search terms." },
-    { "count", notmuch_count_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "count", notmuch_count_command, NOTMUCH_COMMAND_DATABASE_EARLY,
       "Count messages matching the search terms." },
     { "reply", notmuch_reply_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Construct a reply template for a set of messages." },
diff --git a/test/T060-count.sh b/test/T060-count.sh
index a1ebf8ba..39a4dcc5 100755
--- a/test/T060-count.sh
+++ b/test/T060-count.sh
@@ -154,4 +154,15 @@ print("4: {} messages".format(query.count_messages()))
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "count with saved query from config file"
+query_name="test${RANDOM}"
+notmuch count query:$query_name > OUTPUT
+printf "\n[query]\n${query_name} = tag:inbox\n" >> notmuch-config
+notmuch count query:$query_name >> OUTPUT
+cat <<EOF > EXPECTED
+0
+52
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index cef07095..0cacc41d 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -286,6 +286,21 @@ test_begin_subtest "Count, default exclusion: tag in query (threads)"
 output=$(notmuch count --output=threads tag:test and tag:deleted)
 test_expect_equal "$output" "3"
 
+test_begin_subtest "Count, default exclusion, batch"
+notmuch count  --batch --output=messages<<EOF > OUTPUT
+tag:test
+tag:test and tag:deleted
+tag:test
+tag:test and tag:deleted
+EOF
+cat <<EOF >EXPECTED
+2
+4
+2
+4
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Count, exclude=true: tag in query (messages)"
 output=$(notmuch count --exclude=true tag:test and tag:deleted)
 test_expect_equal "$output" "4"
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 13/38] cli/dump: convert to new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (11 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 12/38] CLI/count: switch to new configuration framework David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 14/38] lib: add notmuch_config_get_bool David Bremner
                   ` (24 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This conversion is trivial because the only configuration information
accessed by dump is that stored in the database (in order to dump
it). We do need to be careful to keep the write lock on the database
to ensure dump consistency.
---
 notmuch-dump.c            |  7 +------
 notmuch.c                 |  2 +-
 test/T240-dump-restore.sh | 30 +++++++++++++++++++++++++++++-
 3 files changed, 31 insertions(+), 8 deletions(-)

diff --git a/notmuch-dump.c b/notmuch-dump.c
index eb629dc9..d7017929 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -361,16 +361,11 @@ notmuch_database_dump (notmuch_database_t *notmuch,
 }
 
 int
-notmuch_dump_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_dump_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch , int argc, char *argv[])
 {
-    notmuch_database_t *notmuch;
     const char *query_str = NULL;
     int ret;
 
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
-	return EXIT_FAILURE;
-
     notmuch_exit_if_unmatched_db_uuid (notmuch);
 
     const char *output_file_name = NULL;
diff --git a/notmuch.c b/notmuch.c
index 40527893..b10cc702 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -159,7 +159,7 @@ static command_t commands[] = {
       "Construct a reply template for a set of messages." },
     { "tag", notmuch_tag_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Add/remove tags for all messages matching the search terms." },
-    { "dump", notmuch_dump_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "dump", notmuch_dump_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Create a plain-text dump of the tags for each message." },
     { "restore", notmuch_restore_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Restore the tags from the given dump file (see 'dump')." },
diff --git a/test/T240-dump-restore.sh b/test/T240-dump-restore.sh
index 0870ff92..131daa01 100755
--- a/test/T240-dump-restore.sh
+++ b/test/T240-dump-restore.sh
@@ -82,6 +82,33 @@ test_begin_subtest "dump --output=outfile --"
 notmuch dump --output=dump-1-arg-dash.actual --
 test_expect_equal_file dump.expected dump-1-arg-dash.actual
 
+# configuration
+
+test_begin_subtest "dump with saved query from config file"
+query_name="test${RANDOM}"
+printf "Before:\n" > OUTPUT
+notmuch dump --include=tags query:$query_name | sort >> OUTPUT
+printf "\nAfter:\n" >> OUTPUT
+cp notmuch-config old-config
+printf "\n[query]\n${query_name} = tag:signed\n" >> notmuch-config
+notmuch dump --include=tags query:$query_name | sort >> OUTPUT
+cat <<EOF > EXPECTED
+Before:
+#notmuch-dump batch-tag:3 tags
+
+After:
+#notmuch-dump batch-tag:3 tags
++attachment +inbox +signed +unread -- id:20091118005829.GB25380@dottiness.seas.harvard.edu
++attachment +inbox +signed +unread -- id:20091118010116.GC25380@dottiness.seas.harvard.edu
++inbox +signed +unread -- id:20091117190054.GU3165@dottiness.seas.harvard.edu
++inbox +signed +unread -- id:20091117203301.GV3165@dottiness.seas.harvard.edu
++inbox +signed +unread -- id:20091118002059.067214ed@hikari
++inbox +signed +unread -- id:20091118005040.GA25380@dottiness.seas.harvard.edu
++inbox +signed +unread -- id:87iqd9rn3l.fsf@vertex.dottedmag
+EOF
+cp old-config notmuch-config
+test_expect_equal_file EXPECTED OUTPUT
+
 # gzipped output
 
 test_begin_subtest "dump --gzip"
@@ -322,6 +349,7 @@ EOF
 
 test_expect_equal_file EXPECTED OUTPUT
 
+backup_database
 test_begin_subtest 'roundtripping random message-ids and tags'
 
     ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} \
@@ -338,7 +366,7 @@ test_begin_subtest 'roundtripping random message-ids and tags'
 	 sort > OUTPUT.$test_count
 
 test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+restore_database
 
 test_done
 
-# Note the database is "poisoned" for sup format at this point.
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 14/38] lib: add notmuch_config_get_bool
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (12 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 13/38] cli/dump: convert to new config framework David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 15/38] CLI/restore: convert to new config framework David Bremner
                   ` (23 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Booleans have no out of band values, so return a status for errors.
---
 lib/config.cc | 26 ++++++++++++++++++++++++++
 lib/notmuch.h | 19 +++++++++++++++++++
 2 files changed, 45 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 51e359b8..1318cb52 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -359,6 +359,32 @@ _notmuch_config_load_from_file (notmuch_database_t *notmuch,
     return status;
 }
 
+notmuch_status_t
+notmuch_config_get_bool (notmuch_database_t *notmuch, notmuch_config_key_t key, notmuch_bool_t *val)
+{
+    const char *key_string, *val_string;
+
+    key_string = _notmuch_config_key_to_string (key);
+    if (! key_string) {
+	return NOTMUCH_STATUS_ILLEGAL_ARGUMENT;
+    }
+
+    val_string = _notmuch_string_map_get (notmuch->config, key_string);
+    if (! val_string) {
+	*val = FALSE;
+	return NOTMUCH_STATUS_SUCCESS;
+    }
+
+    if (strcase_equal (val_string, "false") || strcase_equal (val_string, "no"))
+	*val = FALSE;
+    else if (strcase_equal (val_string, "true") || strcase_equal (val_string, "yes"))
+	*val = TRUE;
+    else
+	return NOTMUCH_STATUS_ILLEGAL_ARGUMENT;
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static const char *
 _notmuch_config_key_to_string (notmuch_config_key_t key) {
     switch (key) {
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 39f39423..4a513b44 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2532,6 +2532,25 @@ notmuch_config_values_start (notmuch_config_values_t *values);
 void
 notmuch_config_values_destroy (notmuch_config_values_t *values);
 
+/**
+ * get a configuration value from an open database as Boolean
+ *
+ * This value reflects all configuration information given at the time
+ * the database was opened.
+ *
+ * @param[in] notmuch database
+ * @param[in] key configuration key
+ * @param[out] val configuration value, converted to Boolean
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ *
+ * @retval #NOTMUCH_STATUS_ILLEGAL_ARGUMENT if either key is unknown
+ * or the corresponding value does not convert to Boolean.
+ */
+notmuch_status_t
+notmuch_config_get_bool (notmuch_database_t *notmuch,
+			 notmuch_config_key_t key,
+			 notmuch_bool_t *val);
 /**
  * get the current default indexing options for a given database.
  *
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 15/38] CLI/restore: convert to new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (13 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 14/38] lib: add notmuch_config_get_bool David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 16/38] CLI/insert: " David Bremner
                   ` (22 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Switch one configuration check to new n_c_get_bool function, and
switch use of config as talloc context to notmuch.
---
 notmuch-restore.c | 17 ++++++++++-------
 notmuch.c         |  2 +-
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/notmuch-restore.c b/notmuch-restore.c
index 544f4228..ce07f89d 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -219,9 +219,8 @@ parse_sup_line (void *ctx, char *line,
 }
 
 int
-notmuch_restore_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_restore_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
-    notmuch_database_t *notmuch;
     bool accumulate = false;
     tag_op_flag_t flags = 0;
     tag_op_list_t *tag_ops;
@@ -238,12 +237,16 @@ notmuch_restore_command (notmuch_config_t *config, unused(notmuch_database_t *no
     int include = 0;
     int input_format = DUMP_FORMAT_AUTO;
     int errnum;
+    notmuch_bool_t synchronize_flags;
 
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
+    if (print_status_database (
+	    "notmuch restore",
+	    notmuch,
+	    notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS,
+				     &synchronize_flags)))
 	return EXIT_FAILURE;
 
-    if (notmuch_config_get_maildir_synchronize_flags (config))
+    if (synchronize_flags)
 	flags |= TAG_FLAG_MAILDIR_SYNC;
 
     notmuch_opt_desc_t options[] = {
@@ -310,7 +313,7 @@ notmuch_restore_command (notmuch_config_t *config, unused(notmuch_database_t *no
 	goto DONE;
     }
 
-    tag_ops = tag_op_list_create (config);
+    tag_ops = tag_op_list_create (notmuch);
     if (tag_ops == NULL) {
 	fprintf (stderr, "Out of memory.\n");
 	ret = EXIT_FAILURE;
@@ -377,7 +380,7 @@ notmuch_restore_command (notmuch_config_t *config, unused(notmuch_database_t *no
 	if (line_ctx != NULL)
 	    talloc_free (line_ctx);
 
-	line_ctx = talloc_new (config);
+	line_ctx = talloc_new (notmuch);
 
 	if ((include & DUMP_INCLUDE_PROPERTIES) && line_len >= 2 && line[0] == '#' && line[1] == '=') {
 	    ret = process_properties_line (notmuch, line + 2);
diff --git a/notmuch.c b/notmuch.c
index b10cc702..4258ed43 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -161,7 +161,7 @@ static command_t commands[] = {
       "Add/remove tags for all messages matching the search terms." },
     { "dump", notmuch_dump_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Create a plain-text dump of the tags for each message." },
-    { "restore", notmuch_restore_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "restore", notmuch_restore_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Restore the tags from the given dump file (see 'dump')." },
     { "compact", notmuch_compact_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Compact the notmuch database." },
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 16/38] CLI/insert: convert to new config framework.
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (14 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 15/38] CLI/restore: convert to new config framework David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 17/38] cli/reindex: convert " David Bremner
                   ` (21 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The new talloc context is needed to run the hook at the very end of
the function. That in turn is needed so that this process gives up the
write lock on the database.
---
 notmuch-insert.c    | 59 +++++++++++++++++++++++++++------------------
 notmuch.c           |  2 +-
 test/T070-insert.sh |  8 +++---
 3 files changed, 41 insertions(+), 28 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index de160309..e483b949 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -444,14 +444,12 @@ add_file (notmuch_database_t *notmuch, const char *path, tag_op_list_t *tag_ops,
 }
 
 int
-notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *notmuch, int argc, char *argv[])
 {
     notmuch_status_t status, close_status;
-    notmuch_database_t *notmuch;
     struct sigaction action;
     const char *db_path;
-    const char **new_tags;
-    size_t new_tags_length;
+    notmuch_config_values_t *new_tags = NULL;
     tag_op_list_t *tag_ops;
     char *query_string = NULL;
     const char *folder = "";
@@ -459,11 +457,11 @@ notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *not
     bool keep = false;
     bool hooks = true;
     bool world_readable = false;
-    bool synchronize_flags;
+    notmuch_bool_t synchronize_flags;
     char *maildir;
     char *newpath;
     int opt_index;
-    unsigned int i;
+    void *local = talloc_new (NULL);
 
     notmuch_opt_desc_t options[] = {
 	{ .opt_string = &folder, .name = "folder", .allow_empty = true },
@@ -482,30 +480,46 @@ notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *not
 
     notmuch_process_shared_options (argv[0]);
 
-    db_path = notmuch_config_get_database_path (config);
-    new_tags = notmuch_config_get_new_tags (config, &new_tags_length);
-    synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config);
 
-    tag_ops = tag_op_list_create (config);
+    /* XXX TODO replace this use of DATABASE_PATH with something specific to hooks */
+    db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH);
+
+    if (! db_path)
+	INTERNAL_ERROR ("Unable to retrieve database path");
+    else
+	db_path = talloc_strdup (local, db_path);
+
+    new_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_TAGS);
+
+    if (print_status_database (
+	    "notmuch insert",
+	    notmuch,
+	    notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS,
+				     &synchronize_flags)))
+	return EXIT_FAILURE;
+
+    tag_ops = tag_op_list_create (local);
     if (tag_ops == NULL) {
 	fprintf (stderr, "Out of memory.\n");
 	return EXIT_FAILURE;
     }
-    for (i = 0; i < new_tags_length; i++) {
+    for (;
+	 notmuch_config_values_valid (new_tags);
+	 notmuch_config_values_move_to_next (new_tags)) {
 	const char *error_msg;
-
-	error_msg = illegal_tag (new_tags[i], false);
+	const char *tag = notmuch_config_values_get (new_tags);
+	error_msg = illegal_tag (tag, false);
 	if (error_msg) {
 	    fprintf (stderr, "Error: tag '%s' in new.tags: %s\n",
-		     new_tags[i],  error_msg);
+		     tag,  error_msg);
 	    return EXIT_FAILURE;
 	}
 
-	if (tag_op_list_append (tag_ops, new_tags[i], false))
+	if (tag_op_list_append (tag_ops, tag, false))
 	    return EXIT_FAILURE;
     }
 
-    if (parse_tag_command_line (config, argc - opt_index, argv + opt_index,
+    if (parse_tag_command_line (local, argc - opt_index, argv + opt_index,
 				&query_string, tag_ops))
 	return EXIT_FAILURE;
 
@@ -519,14 +533,14 @@ notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *not
 	return EXIT_FAILURE;
     }
 
-    maildir = talloc_asprintf (config, "%s/%s", db_path, folder);
+    maildir = talloc_asprintf (local, "%s/%s", db_path, folder);
     if (! maildir) {
 	fprintf (stderr, "Out of memory\n");
 	return EXIT_FAILURE;
     }
 
     strip_trailing (maildir, '/');
-    if (create_folder && ! maildir_create_folder (config, maildir, world_readable))
+    if (create_folder && ! maildir_create_folder (local, maildir, world_readable))
 	return EXIT_FAILURE;
 
     /* Set up our handler for SIGINT. We do not set SA_RESTART so that copying
@@ -538,16 +552,11 @@ notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *not
     sigaction (SIGINT, &action, NULL);
 
     /* Write the message to the Maildir new directory. */
-    newpath = maildir_write_new (config, STDIN_FILENO, maildir, world_readable);
+    newpath = maildir_write_new (local, STDIN_FILENO, maildir, world_readable);
     if (! newpath) {
 	return EXIT_FAILURE;
     }
 
-    status = notmuch_database_open (notmuch_config_get_database_path (config),
-				    NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch);
-    if (status)
-	return keep ? NOTMUCH_STATUS_SUCCESS : status_to_exit (status);
-
     notmuch_exit_if_unmatched_db_uuid (notmuch);
 
     status = notmuch_process_shared_indexing_options (notmuch);
@@ -589,5 +598,7 @@ notmuch_insert_command (notmuch_config_t *config, unused(notmuch_database_t *not
 	notmuch_run_hook (db_path, "post-insert");
     }
 
+    talloc_free (local);
+
     return status_to_exit (status);
 }
diff --git a/notmuch.c b/notmuch.c
index 4258ed43..8e697396 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -145,7 +145,7 @@ static command_t commands[] = {
       "Interactively set up notmuch for first use." },
     { "new", notmuch_new_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Find and import new messages to the notmuch database." },
-    { "insert", notmuch_insert_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "insert", notmuch_insert_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Add a new message into the maildir and notmuch database." },
     { "search", notmuch_search_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Search for messages matching the given search terms." },
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index 1c7ca846..7341810c 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -226,11 +226,13 @@ test_expect_code 1 "notmuch insert --folder=../G --create-folder < $gen_msg_file
 
 OLDCONFIG=$(notmuch config get new.tags)
 
-test_begin_subtest "Empty tags in new.tags are forbidden"
+test_begin_subtest "Empty tags in new.tags are ignored"
 notmuch config set new.tags "foo;;bar"
 gen_insert_msg
-output=$(notmuch insert < $gen_msg_filename 2>&1)
-test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden"
+notmuch insert < $gen_msg_filename
+output=$(notmuch show --format=json id:$gen_msg_id)
+test_json_nodes <<<"$output" \
+		'new_tags:[0][0][0]["tags"] = ["bar", "foo"]'
 
 test_begin_subtest "Tags starting with '-' in new.tags are forbidden"
 notmuch config set new.tags "-foo;bar"
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 17/38] cli/reindex: convert new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (15 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 16/38] CLI/insert: " David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 18/38] CLI/reply: convert to " David Bremner
                   ` (20 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The only non-trivial part is switching the talloc context for
query_string_from args from 'config' to 'notmuch'.
---
 notmuch-reindex.c    |  9 ++-------
 notmuch.c            |  2 +-
 test/T700-reindex.sh | 10 ++++++++++
 3 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/notmuch-reindex.c b/notmuch-reindex.c
index 9d337c48..fa84d4fc 100644
--- a/notmuch-reindex.c
+++ b/notmuch-reindex.c
@@ -83,10 +83,9 @@ reindex_query (notmuch_database_t *notmuch, const char *query_string,
 }
 
 int
-notmuch_reindex_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_reindex_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
     char *query_string = NULL;
-    notmuch_database_t *notmuch;
     struct sigaction action;
     int opt_index;
     int ret;
@@ -111,10 +110,6 @@ notmuch_reindex_command (notmuch_config_t *config, unused(notmuch_database_t *no
 
     notmuch_process_shared_options (argv[0]);
 
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
-	return EXIT_FAILURE;
-
     notmuch_exit_if_unmatched_db_uuid (notmuch);
 
     status = notmuch_process_shared_indexing_options (notmuch);
@@ -124,7 +119,7 @@ notmuch_reindex_command (notmuch_config_t *config, unused(notmuch_database_t *no
 	return EXIT_FAILURE;
     }
 
-    query_string = query_string_from_args (config, argc - opt_index, argv + opt_index);
+    query_string = query_string_from_args (notmuch, argc - opt_index, argv + opt_index);
     if (query_string == NULL) {
 	fprintf (stderr, "Out of memory\n");
 	return EXIT_FAILURE;
diff --git a/notmuch.c b/notmuch.c
index 8e697396..82d68681 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -165,7 +165,7 @@ static command_t commands[] = {
       "Restore the tags from the given dump file (see 'dump')." },
     { "compact", notmuch_compact_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Compact the notmuch database." },
-    { "reindex", notmuch_reindex_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "reindex", notmuch_reindex_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Re-index all messages matching the search terms." },
     { "config", notmuch_config_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Get or set settings in the notmuch configuration file." },
diff --git a/test/T700-reindex.sh b/test/T700-reindex.sh
index 3d7c930d..67ee8452 100755
--- a/test/T700-reindex.sh
+++ b/test/T700-reindex.sh
@@ -75,6 +75,16 @@ notmuch reindex '*'
 notmuch dump | grep '^#=' | sort > OUTPUT
 test_expect_equal_file prop-dump OUTPUT
 
+test_begin_subtest "reindex with saved query from config file"
+query_name="test${RANDOM}"
+count1=$(notmuch count --lastmod '*' | cut -f3)
+cp notmuch-config old-config
+printf "\n[query]\n${query_name} = tag:inbox\n" >> notmuch-config
+notmuch reindex query:$query_name
+count2=$(notmuch count --lastmod '*' | cut -f3)
+cp old-config notmuch-config
+test_expect_success "test '$count2 -gt $count1'"
+
 add_email_corpus lkml
 
 test_begin_subtest "reindex of lkml corpus preserves threads"
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 18/38] CLI/reply: convert to new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (16 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 17/38] cli/reindex: convert " David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 19/38] CLI/{search,address}: convert to new configuration framework David Bremner
                   ` (19 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is messier than some of the other conversions because the
extensive use of 'config' as a talloc context.
---
 notmuch-reply.c    | 100 ++++++++++++++++++++++-----------------------
 notmuch.c          |   2 +-
 test/T220-reply.sh |  24 +++++++++++
 3 files changed, 74 insertions(+), 52 deletions(-)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index a8ffbf75..700f3397 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -112,25 +112,26 @@ match_address (const char *str, const char *address, address_match_t mode)
 /* Match given string against user's configured "primary" and "other"
  * addresses according to mode. */
 static const char *
-address_match (const char *str, notmuch_config_t *config, address_match_t mode)
+address_match (const char *str, notmuch_database_t *notmuch, address_match_t mode)
 {
     const char *primary;
-    const char **other;
-    size_t i, other_len;
+    notmuch_config_values_t *other = NULL;
 
     if (! str || *str == '\0')
 	return NULL;
 
-    primary = notmuch_config_get_user_primary_email (config);
+    primary = notmuch_config_get (notmuch, NOTMUCH_CONFIG_PRIMARY_EMAIL);
     if (match_address (str, primary, mode))
 	return primary;
 
-    other = notmuch_config_get_user_other_email (config, &other_len);
-    for (i = 0; i < other_len; i++) {
-	if (match_address (str, other[i], mode))
-	    return other[i];
-    }
+    for (other = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_OTHER_EMAIL);
+	 notmuch_config_values_valid (other);
+	 notmuch_config_values_move_to_next (other)) {
+	const char *addr = notmuch_config_values_get (other);
 
+	if (match_address (str, addr, mode))
+	    return addr;
+    }
     return NULL;
 }
 
@@ -138,26 +139,26 @@ address_match (const char *str, notmuch_config_t *config, address_match_t mode)
  * user's "primary" or "other" addresses. If so, return the matching
  * address, NULL otherwise. */
 static const char *
-user_address_in_string (const char *str, notmuch_config_t *config)
+user_address_in_string (const char *str, notmuch_database_t *notmuch)
 {
-    return address_match (str, config, USER_ADDRESS_IN_STRING);
+    return address_match (str, notmuch, USER_ADDRESS_IN_STRING);
 }
 
 /* Do any of the addresses configured as one of the user's "primary"
  * or "other" addresses contain the given string. If so, return the
  * matching address, NULL otherwise. */
 static const char *
-string_in_user_address (const char *str, notmuch_config_t *config)
+string_in_user_address (const char *str, notmuch_database_t *notmuch)
 {
-    return address_match (str, config, STRING_IN_USER_ADDRESS);
+    return address_match (str, notmuch, STRING_IN_USER_ADDRESS);
 }
 
 /* Is the given address configured as one of the user's "primary" or
  * "other" addresses. */
 static bool
-address_is_users (const char *address, notmuch_config_t *config)
+address_is_users (const char *address, notmuch_database_t *notmuch)
 {
-    return address_match (address, config, STRING_IS_USER_ADDRESS) != NULL;
+    return address_match (address, notmuch, STRING_IS_USER_ADDRESS) != NULL;
 }
 
 /* Scan addresses in 'list'.
@@ -175,7 +176,7 @@ address_is_users (const char *address, notmuch_config_t *config)
  */
 static unsigned int
 scan_address_list (InternetAddressList *list,
-		   notmuch_config_t *config,
+		   notmuch_database_t *notmuch,
 		   GMimeMessage *message,
 		   GMimeAddressType type,
 		   const char **user_from)
@@ -195,7 +196,7 @@ scan_address_list (InternetAddressList *list,
 
 	    group = INTERNET_ADDRESS_GROUP (address);
 	    group_list = internet_address_group_get_members (group);
-	    n += scan_address_list (group_list, config, message, type, user_from);
+	    n += scan_address_list (group_list, notmuch, message, type, user_from);
 	} else {
 	    InternetAddressMailbox *mailbox;
 	    const char *name;
@@ -206,7 +207,7 @@ scan_address_list (InternetAddressList *list,
 	    name = internet_address_get_name (address);
 	    addr = internet_address_mailbox_get_addr (mailbox);
 
-	    if (address_is_users (addr, config)) {
+	    if (address_is_users (addr, notmuch)) {
 		if (user_from && *user_from == NULL)
 		    *user_from = addr;
 	    } else if (message) {
@@ -324,7 +325,7 @@ get_bcc (GMimeMessage *message)
  */
 static const char *
 add_recipients_from_message (GMimeMessage *reply,
-			     notmuch_config_t *config,
+			     notmuch_database_t *notmuch,
 			     GMimeMessage *message,
 			     bool reply_all)
 {
@@ -346,7 +347,7 @@ add_recipients_from_message (GMimeMessage *reply,
 
 	recipients = reply_to_map[i].get_header (message);
 
-	n += scan_address_list (recipients, config, reply,
+	n += scan_address_list (recipients, notmuch, reply,
 				reply_to_map[i].recipient_type, &from_addr);
 
 	if (! reply_all && n) {
@@ -384,7 +385,7 @@ add_recipients_from_message (GMimeMessage *reply,
  * Return the address that was found, if any, and NULL otherwise.
  */
 static const char *
-guess_from_in_received_for (notmuch_config_t *config, const char *received)
+guess_from_in_received_for (notmuch_database_t *notmuch, const char *received)
 {
     const char *ptr;
 
@@ -392,7 +393,7 @@ guess_from_in_received_for (notmuch_config_t *config, const char *received)
     if (! ptr)
 	return NULL;
 
-    return user_address_in_string (ptr, config);
+    return user_address_in_string (ptr, notmuch);
 }
 
 /*
@@ -408,7 +409,7 @@ guess_from_in_received_for (notmuch_config_t *config, const char *received)
  * Return the address that was found, if any, and NULL otherwise.
  */
 static const char *
-guess_from_in_received_by (notmuch_config_t *config, const char *received)
+guess_from_in_received_by (notmuch_database_t *notmuch, const char *received)
 {
     const char *addr;
     const char *by = received;
@@ -446,7 +447,7 @@ guess_from_in_received_by (notmuch_config_t *config, const char *received)
 	     */
 	    *(tld - 1) = '.';
 
-	    addr = string_in_user_address (domain, config);
+	    addr = string_in_user_address (domain, notmuch);
 	    if (addr) {
 		free (mta);
 		return addr;
@@ -469,12 +470,13 @@ guess_from_in_received_by (notmuch_config_t *config, const char *received)
  * Return the address that was found, if any, and NULL otherwise.
  */
 static const char *
-guess_from_in_received_headers (notmuch_config_t *config,
-				notmuch_message_t *message)
+guess_from_in_received_headers (notmuch_message_t *message)
 {
     const char *received, *addr;
     char *sanitized;
 
+    notmuch_database_t *notmuch = notmuch_message_get_database (message);
+
     received = notmuch_message_get_header (message, "received");
     if (! received)
 	return NULL;
@@ -483,9 +485,9 @@ guess_from_in_received_headers (notmuch_config_t *config,
     if (! sanitized)
 	return NULL;
 
-    addr = guess_from_in_received_for (config, sanitized);
+    addr = guess_from_in_received_for (notmuch, sanitized);
     if (! addr)
-	addr = guess_from_in_received_by (config, sanitized);
+	addr = guess_from_in_received_by (notmuch, sanitized);
 
     talloc_free (sanitized);
 
@@ -500,7 +502,7 @@ guess_from_in_received_headers (notmuch_config_t *config,
  * Return the address that was found, if any, and NULL otherwise.
  */
 static const char *
-get_from_in_to_headers (notmuch_config_t *config, notmuch_message_t *message)
+get_from_in_to_headers (notmuch_message_t *message)
 {
     size_t i;
     const char *tohdr, *addr;
@@ -510,11 +512,13 @@ get_from_in_to_headers (notmuch_config_t *config, notmuch_message_t *message)
 	"Delivered-To",
     };
 
+    notmuch_database_t *notmuch = notmuch_message_get_database (message);
+
     for (i = 0; i < ARRAY_SIZE (to_headers); i++) {
 	tohdr = notmuch_message_get_header (message, to_headers[i]);
 
 	/* Note: tohdr potentially contains a list of email addresses. */
-	addr = user_address_in_string (tohdr, config);
+	addr = user_address_in_string (tohdr, notmuch);
 	if (addr)
 	    return addr;
     }
@@ -524,7 +528,6 @@ get_from_in_to_headers (notmuch_config_t *config, notmuch_message_t *message)
 
 static GMimeMessage *
 create_reply_message (void *ctx,
-		      notmuch_config_t *config,
 		      notmuch_message_t *message,
 		      GMimeMessage *mime_message,
 		      bool reply_all,
@@ -532,7 +535,7 @@ create_reply_message (void *ctx,
 {
     const char *subject, *from_addr = NULL;
     const char *in_reply_to, *orig_references, *references;
-
+    notmuch_database_t *notmuch = notmuch_message_get_database (message);
     /*
      * Use the below header order for limited headers, "pretty" order
      * otherwise.
@@ -558,7 +561,7 @@ create_reply_message (void *ctx,
 
     g_mime_object_set_header (GMIME_OBJECT (reply), "References", references, NULL);
 
-    from_addr = add_recipients_from_message (reply, config,
+    from_addr = add_recipients_from_message (reply, notmuch,
 					     mime_message, reply_all);
 
     /* The above is all that is needed for limited headers. */
@@ -578,7 +581,7 @@ create_reply_message (void *ctx,
      * Delivered-To: headers.
      */
     if (from_addr == NULL)
-	from_addr = get_from_in_to_headers (config, message);
+	from_addr = get_from_in_to_headers (message);
 
     /*
      * Check for a (for <email@add.res>) clause in Received: headers,
@@ -586,14 +589,14 @@ create_reply_message (void *ctx,
      * of Received: headers
      */
     if (from_addr == NULL)
-	from_addr = guess_from_in_received_headers (config, message);
+	from_addr = guess_from_in_received_headers (message);
 
     /* Default to user's primary address. */
     if (from_addr == NULL)
-	from_addr = notmuch_config_get_user_primary_email (config);
+	from_addr = notmuch_config_get (notmuch, NOTMUCH_CONFIG_PRIMARY_EMAIL);
 
     from_addr = talloc_asprintf (ctx, "%s <%s>",
-				 notmuch_config_get_user_name (config),
+				 notmuch_config_get (notmuch, NOTMUCH_CONFIG_USER_NAME),
 				 from_addr);
     g_mime_object_set_header (GMIME_OBJECT (reply), "From", from_addr, NULL);
 
@@ -615,7 +618,7 @@ enum {
 };
 
 static int
-do_reply (notmuch_config_t *config,
+do_reply (notmuch_database_t *notmuch,
 	  notmuch_query_t *query,
 	  notmuch_show_params_t *params,
 	  int format,
@@ -641,9 +644,9 @@ do_reply (notmuch_config_t *config,
 	}
 
 	if (format == FORMAT_JSON)
-	    sp = sprinter_json_create (config, stdout);
+	    sp = sprinter_json_create (notmuch, stdout);
 	else
-	    sp = sprinter_sexp_create (config, stdout);
+	    sp = sprinter_sexp_create (notmuch, stdout);
     }
 
     status = notmuch_query_search_messages (query, &messages);
@@ -655,10 +658,10 @@ do_reply (notmuch_config_t *config,
 	 notmuch_messages_move_to_next (messages)) {
 	message = notmuch_messages_get (messages);
 
-	if (mime_node_open (config, message, &params->crypto, &node))
+	if (mime_node_open (notmuch, message, &params->crypto, &node))
 	    return 1;
 
-	reply = create_reply_message (config, config, message,
+	reply = create_reply_message (notmuch, message,
 				      GMIME_MESSAGE (node->part), reply_all,
 				      format == FORMAT_HEADERS_ONLY);
 	if (! reply)
@@ -675,7 +678,7 @@ do_reply (notmuch_config_t *config,
 
 	    /* Start the original */
 	    sp->map_key (sp, "original");
-	    format_part_sprinter (config, sp, node, true, false);
+	    format_part_sprinter (notmuch, sp, node, true, false);
 
 	    /* End */
 	    sp->end (sp);
@@ -700,9 +703,8 @@ do_reply (notmuch_config_t *config,
 }
 
 int
-notmuch_reply_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_reply_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
-    notmuch_database_t *notmuch;
     notmuch_query_t *query;
     char *query_string;
     int opt_index;
@@ -743,7 +745,7 @@ notmuch_reply_command (notmuch_config_t *config, unused(notmuch_database_t *notm
 
     notmuch_exit_if_unsupported_format ();
 
-    query_string = query_string_from_args (config, argc - opt_index, argv + opt_index);
+    query_string = query_string_from_args (notmuch, argc - opt_index, argv + opt_index);
     if (query_string == NULL) {
 	fprintf (stderr, "Out of memory\n");
 	return EXIT_FAILURE;
@@ -754,10 +756,6 @@ notmuch_reply_command (notmuch_config_t *config, unused(notmuch_database_t *notm
 	return EXIT_FAILURE;
     }
 
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_ONLY, &notmuch))
-	return EXIT_FAILURE;
-
     notmuch_exit_if_unmatched_db_uuid (notmuch);
 
     query = notmuch_query_create (notmuch, query_string);
@@ -766,7 +764,7 @@ notmuch_reply_command (notmuch_config_t *config, unused(notmuch_database_t *notm
 	return EXIT_FAILURE;
     }
 
-    if (do_reply (config, query, &params, format, reply_all) != 0)
+    if (do_reply (notmuch, query, &params, format, reply_all) != 0)
 	return EXIT_FAILURE;
 
     _notmuch_crypto_cleanup (&params.crypto);
diff --git a/notmuch.c b/notmuch.c
index 82d68681..95cd4ae5 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -155,7 +155,7 @@ static command_t commands[] = {
       "Show all messages matching the search terms." },
     { "count", notmuch_count_command, NOTMUCH_COMMAND_DATABASE_EARLY,
       "Count messages matching the search terms." },
-    { "reply", notmuch_reply_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "reply", notmuch_reply_command, NOTMUCH_COMMAND_DATABASE_EARLY,
       "Construct a reply template for a set of messages." },
     { "tag", notmuch_tag_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Add/remove tags for all messages matching the search terms." },
diff --git a/test/T220-reply.sh b/test/T220-reply.sh
index b6d8f42a..791b9d6e 100755
--- a/test/T220-reply.sh
+++ b/test/T220-reply.sh
@@ -58,6 +58,30 @@ On Tue, 05 Jan 2010 15:43:56 -0000, Sender <sender@example.com> wrote:
 > reply with CC
 OK"
 
+test_begin_subtest "reply with saved query from config file"
+query_name="test${RANDOM}"
+printf "Before:\n" > OUTPUT
+notmuch reply query:$query_name 2>&1 >> OUTPUT
+cp notmuch-config old-config
+printf "\n[query]\n${query_name} = id:${gen_msg_id}\n" >> notmuch-config
+printf "After:\n" >> OUTPUT
+notmuch reply query:$query_name >> OUTPUT
+cat <<EOF > EXPECTED
+Before:
+After:
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+Subject: Re: notmuch-reply-test
+To: Sender <sender@example.com>
+Cc: Other Parties <cc@example.com>
+In-Reply-To: <msg-003@notmuch-test-suite>
+References: <msg-003@notmuch-test-suite>
+
+On Tue, 05 Jan 2010 15:43:56 -0000, Sender <sender@example.com> wrote:
+> reply with CC
+EOF
+cp old-config notmuch-config
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Reply from alternate address"
 add_message '[from]="Sender <sender@example.com>"' \
 	     [to]=test_suite_other@notmuchmail.org \
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 19/38] CLI/{search,address}: convert to new configuration framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (17 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 18/38] CLI/reply: convert to " David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 20/38] cli/config: add accessor for config file name David Bremner
                   ` (18 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Since we are already passing a search context around as a kind of
parameter block, add a new talloc context to that to replace relying
on 'config'.

Convert notmuch-search and notmuch-address at the same time, because
they share some code.

Add a test to make sure we don't break passing configuration as a
command line argument.
---
 notmuch-search.c         | 54 ++++++++++++++++------------------------
 notmuch.c                |  4 +--
 test/T080-search.sh      | 35 ++++++++++++++++++++++++++
 test/T095-address.sh     | 22 ++++++++++++++++
 test/T750-user-header.sh | 18 ++++++++++++++
 5 files changed, 99 insertions(+), 34 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 34e27058..aba22799 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -52,6 +52,7 @@ typedef enum {
 
 typedef struct {
     notmuch_database_t *notmuch;
+    void *talloc_ctx;
     int format_sel;
     sprinter_t *format;
     int exclude;
@@ -677,28 +678,29 @@ do_search_tags (const search_context_t *ctx)
 }
 
 static int
-_notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int argc, char *argv[])
+_notmuch_search_prepare (search_context_t *ctx, int argc, char *argv[])
 {
     char *query_str;
-    unsigned int i;
-    char *status_string = NULL;
+
+    if (! ctx->talloc_ctx)
+	ctx->talloc_ctx = talloc_new (NULL);
 
     switch (ctx->format_sel) {
     case NOTMUCH_FORMAT_TEXT:
-	ctx->format = sprinter_text_create (config, stdout);
+	ctx->format = sprinter_text_create (ctx->talloc_ctx, stdout);
 	break;
     case NOTMUCH_FORMAT_TEXT0:
 	if (ctx->output == OUTPUT_SUMMARY) {
 	    fprintf (stderr, "Error: --format=text0 is not compatible with --output=summary.\n");
 	    return EXIT_FAILURE;
 	}
-	ctx->format = sprinter_text0_create (config, stdout);
+	ctx->format = sprinter_text0_create (ctx->talloc_ctx, stdout);
 	break;
     case NOTMUCH_FORMAT_JSON:
-	ctx->format = sprinter_json_create (config, stdout);
+	ctx->format = sprinter_json_create (ctx->talloc_ctx, stdout);
 	break;
     case NOTMUCH_FORMAT_SEXP:
-	ctx->format = sprinter_sexp_create (config, stdout);
+	ctx->format = sprinter_sexp_create (ctx->talloc_ctx, stdout);
 	break;
     default:
 	/* this should never happen */
@@ -707,18 +709,6 @@ _notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int ar
 
     notmuch_exit_if_unsupported_format ();
 
-    if (notmuch_database_open_verbose (
-	    notmuch_config_get_database_path (config),
-	    NOTMUCH_DATABASE_MODE_READ_ONLY, &ctx->notmuch, &status_string)) {
-
-	if (status_string) {
-	    fputs (status_string, stderr);
-	    free (status_string);
-	}
-
-	return EXIT_FAILURE;
-    }
-
     notmuch_exit_if_unmatched_db_uuid (ctx->notmuch);
 
     query_str = query_string_from_args (ctx->notmuch, argc, argv);
@@ -748,21 +738,20 @@ _notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int ar
     }
 
     if (ctx->exclude != NOTMUCH_EXCLUDE_FALSE) {
-	const char **search_exclude_tags;
-	size_t search_exclude_tags_length;
+	notmuch_config_values_t *exclude_tags;
 	notmuch_status_t status;
 
-	search_exclude_tags = notmuch_config_get_search_exclude_tags (
-	    config, &search_exclude_tags_length);
+	for (exclude_tags = notmuch_config_get_values (ctx->notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS);
+	     notmuch_config_values_valid (exclude_tags);
+	     notmuch_config_values_move_to_next (exclude_tags)) {
 
-	for (i = 0; i < search_exclude_tags_length; i++) {
-	    status = notmuch_query_add_tag_exclude (ctx->query, search_exclude_tags[i]);
+	    status = notmuch_query_add_tag_exclude (ctx->query,
+						    notmuch_config_values_get (exclude_tags));
 	    if (status && status != NOTMUCH_STATUS_IGNORED) {
 		print_status_query ("notmuch search", ctx->query, status);
 		return EXIT_FAILURE;
 	    }
 	}
-
 	notmuch_query_set_omit_excluded (ctx->query, ctx->exclude);
     }
 
@@ -805,7 +794,7 @@ static const notmuch_opt_desc_t common_options[] = {
 };
 
 int
-notmuch_search_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_search_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
     search_context_t *ctx = &search_context;
     int opt_index, ret;
@@ -832,6 +821,7 @@ notmuch_search_command (notmuch_config_t *config, unused(notmuch_database_t *not
 	{ }
     };
 
+    ctx->notmuch = notmuch;
     ctx->output = OUTPUT_SUMMARY;
     opt_index = parse_arguments (argc, argv, options, 1);
     if (opt_index < 0)
@@ -845,8 +835,7 @@ notmuch_search_command (notmuch_config_t *config, unused(notmuch_database_t *not
 	return EXIT_FAILURE;
     }
 
-    if (_notmuch_search_prepare (ctx, config,
-				 argc - opt_index, argv + opt_index))
+    if (_notmuch_search_prepare (ctx, argc - opt_index, argv + opt_index))
 	return EXIT_FAILURE;
 
     switch (ctx->output) {
@@ -871,7 +860,7 @@ notmuch_search_command (notmuch_config_t *config, unused(notmuch_database_t *not
 }
 
 int
-notmuch_address_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_address_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
     search_context_t *ctx = &search_context;
     int opt_index, ret;
@@ -897,6 +886,8 @@ notmuch_address_command (notmuch_config_t *config, unused(notmuch_database_t *no
 	{ }
     };
 
+    ctx->notmuch = notmuch;
+
     opt_index = parse_arguments (argc, argv, options, 1);
     if (opt_index < 0)
 	return EXIT_FAILURE;
@@ -911,8 +902,7 @@ notmuch_address_command (notmuch_config_t *config, unused(notmuch_database_t *no
 	return EXIT_FAILURE;
     }
 
-    if (_notmuch_search_prepare (ctx, config,
-				 argc - opt_index, argv + opt_index))
+    if (_notmuch_search_prepare (ctx, argc - opt_index, argv + opt_index))
 	return EXIT_FAILURE;
 
     ctx->addresses = g_hash_table_new_full (strcase_hash, strcase_equal,
diff --git a/notmuch.c b/notmuch.c
index 95cd4ae5..16504fc5 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -147,9 +147,9 @@ static command_t commands[] = {
       "Find and import new messages to the notmuch database." },
     { "insert", notmuch_insert_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Add a new message into the maildir and notmuch database." },
-    { "search", notmuch_search_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "search", notmuch_search_command, NOTMUCH_COMMAND_DATABASE_EARLY,
       "Search for messages matching the given search terms." },
-    { "address", notmuch_address_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "address", notmuch_address_command, NOTMUCH_COMMAND_DATABASE_EARLY,
       "Get addresses from messages matching the given search terms." },
     { "show", notmuch_show_command, NOTMUCH_COMMAND_CONFIG_OPEN,
       "Show all messages matching the search terms." },
diff --git a/test/T080-search.sh b/test/T080-search.sh
index a3f0dead..8e82cf55 100755
--- a/test/T080-search.sh
+++ b/test/T080-search.sh
@@ -189,4 +189,39 @@ test_begin_subtest "parts do not have adjacent term positions"
 output=$(notmuch search id:termpos and '"c x"')
 test_expect_equal "$output" ""
 
+backup_database
+test_begin_subtest "search with alternate config"
+cp notmuch-config alt-config
+notmuch --config=alt-config config set search.exclude_tags foobar17
+notmuch tag -- +foobar17 '*'
+output=$(notmuch --config=alt-config count '*')
+test_expect_equal "$output" "0"
+restore_database
+
+test_begin_subtest "search with saved query from config file"
+query_name="test${RANDOM}"
+printf "Before:\n" > OUTPUT
+notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT
+cp notmuch-config old-config
+printf "\n[query]\n${query_name} = from:cworth\n" >> notmuch-config
+printf "After:\n" >> OUTPUT
+notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT
+cat <<EOF > EXPECTED
+Before:
+After:
+thread:XXX   2009-11-18 [1/2] Carl Worth| Alex Botero-Lowry; [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread)
+thread:XXX   2009-11-18 [1/2] Carl Worth| Ingmar Vanhassel; [notmuch] [PATCH] Typsos (inbox unread)
+thread:XXX   2009-11-18 [1/3] Carl Worth| Adrian Perez de Castro, Keith Packard; [notmuch] Introducing myself (inbox signed unread)
+thread:XXX   2009-11-18 [1/3] Carl Worth| Israel Herraiz, Keith Packard; [notmuch] New to the list (inbox unread)
+thread:XXX   2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread)
+thread:XXX   2009-11-18 [1/2] Carl Worth| Jan Janak; [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread)
+thread:XXX   2009-11-18 [1/3(4)] Carl Worth| Aron Griffis, Keith Packard; [notmuch] archive (inbox unread)
+thread:XXX   2009-11-18 [1/2] Carl Worth| Keith Packard; [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread)
+thread:XXX   2009-11-18 [1/7] Carl Worth| Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard; [notmuch] Working with Maildir storage? (inbox signed unread)
+thread:XXX   2009-11-18 [2/5] Carl Worth| Mikhail Gusarov, Keith Packard; [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread)
+thread:XXX   2009-11-17 [1/2] Carl Worth| Alex Botero-Lowry; [notmuch] preliminary FreeBSD support (attachment inbox unread)
+EOF
+cp old-config notmuch-config
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
diff --git a/test/T095-address.sh b/test/T095-address.sh
index 817be538..ebd7c167 100755
--- a/test/T095-address.sh
+++ b/test/T095-address.sh
@@ -325,4 +325,26 @@ cat <<EOF >EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "saved query from config file"
+query_name="test${RANDOM}"
+printf "Before:\n" > OUTPUT
+notmuch address --deduplicate=no --output=sender query:$query_name 2>&1 | sort >> OUTPUT
+cp notmuch-config old-config
+printf "\n[query]\n${query_name} = from:foo.bar@example.com\n" >> notmuch-config
+printf "After:\n" >> OUTPUT
+notmuch address --deduplicate=no --output=sender query:$query_name | sort >> OUTPUT
+cat <<EOF > EXPECTED
+Before:
+After:
+Bar <Foo.Bar@Example.Com>
+Foo <foo.bar@example.com>
+Foo Bar <Foo.Bar@Example.Com>
+Foo Bar <foo.bar@example.com>
+Foo Bar <foo.bar@example.com>
+foo.bar@example.com
+foo.bar@example.com
+EOF
+cp old-config notmuch-config
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
diff --git a/test/T750-user-header.sh b/test/T750-user-header.sh
index 204c052a..ff554b06 100755
--- a/test/T750-user-header.sh
+++ b/test/T750-user-header.sh
@@ -108,4 +108,22 @@ MAIL_DIR/new/04:2,
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "index user header, config from file"
+field_name="Test"
+printf "\n[index]\nheader.${field_name} = List-Id\n" >> notmuch-config
+notmuch reindex '*'
+notmuch search --output=files ${field_name}:notmuch | notmuch_search_files_sanitize | sort > OUTPUT
+cat <<EOF > EXPECTED
+MAIL_DIR/bar/baz/05:2,
+MAIL_DIR/bar/baz/23:2,
+MAIL_DIR/bar/baz/24:2,
+MAIL_DIR/bar/cur/20:2,
+MAIL_DIR/bar/new/21:2,
+MAIL_DIR/bar/new/22:2,
+MAIL_DIR/foo/cur/08:2,
+MAIL_DIR/foo/new/03:2,
+MAIL_DIR/new/04:2,
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 20/38] cli/config: add accessor for config file name
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (18 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 19/38] CLI/{search,address}: convert to new configuration framework David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 21/38] CLI/show: mostly switch show to new config framework David Bremner
                   ` (17 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is intended for use in temporary code transitioning to the new
configuration system. The name is chosen to avoid cluttering the
notmuch_config_* namespace further with non-library functions.
---
 notmuch-client.h | 2 ++
 notmuch-config.c | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/notmuch-client.h b/notmuch-client.h
index e8fb0323..a026002a 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -334,6 +334,8 @@ void
 notmuch_config_set_search_exclude_tags (notmuch_config_t *config,
 					const char *list[],
 					size_t length);
+const char *
+_notmuch_config_get_path (notmuch_config_t *config);
 
 int
 notmuch_run_hook (const char *db_path, const char *hook);
diff --git a/notmuch-config.c b/notmuch-config.c
index 4fa274c7..0193401f 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -510,6 +510,9 @@ notmuch_config_close (notmuch_config_t *config)
     talloc_free (config);
 }
 
+const char *_notmuch_config_get_path (notmuch_config_t *config) {
+    return config->filename;
+}
 /* Save any changes made to the notmuch configuration.
  *
  * Any comments originally in the file will be preserved.
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 21/38] CLI/show: mostly switch show to new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (19 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 20/38] cli/config: add accessor for config file name David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 22/38] cli/tag: convert " David Bremner
                   ` (16 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will need some cleanup when the transition completes, and we stop
passing notmuch_config_t structs to the subcommands.

Unlike the general case, we open the database in the subcommand, since
we don't know whether it should be opened read/write until we parse
the command line arguments.

Add a test to make sure passing config file on the command line is not
broken by these or future config related changes.
---
 notmuch-show.c    | 54 +++++++++++++++++++++++++++++------------------
 test/T520-show.sh |  9 ++++++++
 2 files changed, 42 insertions(+), 21 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 04b90cd7..5e90c143 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1234,6 +1234,7 @@ notmuch_show_command (notmuch_config_t *config, unused(notmuch_database_t *notmu
     bool entire_thread_set = false;
     bool single_message;
     bool unthreaded = FALSE;
+    char *status_string = NULL;
 
     notmuch_opt_desc_t options[] = {
 	{ .opt_keyword = &format, .name = "format", .keywords =
@@ -1323,7 +1324,28 @@ notmuch_show_command (notmuch_config_t *config, unused(notmuch_database_t *notmu
 	fprintf (stderr, "Warning: --include-html only implemented for format=text, format=json and format=sexp\n");
     }
 
-    query_string = query_string_from_args (config, argc - opt_index, argv + opt_index);
+    notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_ONLY;
+    if (params.crypto.decrypt == NOTMUCH_DECRYPT_TRUE)
+	mode = NOTMUCH_DATABASE_MODE_READ_WRITE;
+    if (notmuch_database_open_with_config (NULL,
+					   mode,
+					   _notmuch_config_get_path (config),
+					   NULL,
+					   &notmuch,
+					   &status_string)) {
+	if (status_string) {
+	    fputs (status_string, stderr);
+	    free (status_string);
+	}
+
+	return EXIT_FAILURE;
+    }
+
+    config = NULL;
+
+    notmuch_exit_if_unmatched_db_uuid (notmuch);
+
+    query_string = query_string_from_args (notmuch, argc - opt_index, argv + opt_index);
     if (query_string == NULL) {
 	fprintf (stderr, "Out of memory\n");
 	return EXIT_FAILURE;
@@ -1334,15 +1356,6 @@ notmuch_show_command (notmuch_config_t *config, unused(notmuch_database_t *notmu
 	return EXIT_FAILURE;
     }
 
-    notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_ONLY;
-    if (params.crypto.decrypt == NOTMUCH_DECRYPT_TRUE)
-	mode = NOTMUCH_DATABASE_MODE_READ_WRITE;
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       mode, &notmuch))
-	return EXIT_FAILURE;
-
-    notmuch_exit_if_unmatched_db_uuid (notmuch);
-
     query = notmuch_query_create (notmuch, query_string);
     if (query == NULL) {
 	fprintf (stderr, "Out of memory\n");
@@ -1351,27 +1364,26 @@ notmuch_show_command (notmuch_config_t *config, unused(notmuch_database_t *notmu
 
     /* Create structure printer. */
     formatter = formatters[format];
-    sprinter = formatter->new_sprinter (config, stdout);
+    sprinter = formatter->new_sprinter (notmuch, stdout);
 
     params.out_stream = g_mime_stream_stdout_new ();
 
     /* If a single message is requested we do not use search_excludes. */
     if (single_message) {
-	ret = do_show_single (config, query, formatter, sprinter, &params);
+	ret = do_show_single (notmuch, query, formatter, sprinter, &params);
     } else {
 	/* We always apply set the exclude flag. The
 	 * exclude=true|false option controls whether or not we return
 	 * threads that only match in an excluded message */
-	const char **search_exclude_tags;
-	size_t search_exclude_tags_length;
-	unsigned int i;
+	notmuch_config_values_t *exclude_tags;
 	notmuch_status_t status;
 
-	search_exclude_tags = notmuch_config_get_search_exclude_tags
-				  (config, &search_exclude_tags_length);
+	for (exclude_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS);
+	     notmuch_config_values_valid (exclude_tags);
+	     notmuch_config_values_move_to_next (exclude_tags)) {
 
-	for (i = 0; i < search_exclude_tags_length; i++) {
-	    status = notmuch_query_add_tag_exclude (query, search_exclude_tags[i]);
+	    status = notmuch_query_add_tag_exclude (query,
+						    notmuch_config_values_get (exclude_tags));
 	    if (status && status != NOTMUCH_STATUS_IGNORED) {
 		print_status_query ("notmuch show", query, status);
 		ret = -1;
@@ -1385,9 +1397,9 @@ notmuch_show_command (notmuch_config_t *config, unused(notmuch_database_t *notmu
 	}
 
 	if (unthreaded)
-	    ret = do_show_unthreaded (config, query, formatter, sprinter, &params);
+	    ret = do_show_unthreaded (notmuch, query, formatter, sprinter, &params);
 	else
-	    ret = do_show_threaded (config, query, formatter, sprinter, &params);
+	    ret = do_show_threaded (notmuch, query, formatter, sprinter, &params);
     }
 
   DONE:
diff --git a/test/T520-show.sh b/test/T520-show.sh
index 16222650..e718a18c 100755
--- a/test/T520-show.sh
+++ b/test/T520-show.sh
@@ -10,4 +10,13 @@ notmuch show foo..
 exit_code=$?
 test_expect_equal 1 $exit_code
 
+backup_database
+test_begin_subtest "show with alternate config"
+cp notmuch-config alt-config
+notmuch --config=alt-config config set search.exclude_tags foobar17
+notmuch tag -- +foobar17 '*'
+output=$(notmuch --config=alt-config show '*')
+test_expect_equal "$output" ""
+restore_database
+
 test_done
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 22/38] cli/tag: convert to new config framework.
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (20 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 21/38] CLI/show: mostly switch show to new config framework David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 23/38] lib/config: add _notmuch_config_cache David Bremner
                   ` (15 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

In addition to changing configuration access, change talloc context
for allocation.
---
 notmuch-tag.c | 25 ++++++++++++++-----------
 notmuch.c     |  2 +-
 2 files changed, 15 insertions(+), 12 deletions(-)

diff --git a/notmuch-tag.c b/notmuch-tag.c
index 205f2733..464874b4 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -187,11 +187,10 @@ tag_file (void *ctx, notmuch_database_t *notmuch, tag_op_flag_t flags,
 }
 
 int
-notmuch_tag_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_tag_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
     tag_op_list_t *tag_ops = NULL;
     char *query_string = NULL;
-    notmuch_database_t *notmuch;
     struct sigaction action;
     tag_op_flag_t tag_flags = TAG_FLAG_NONE;
     bool batch = false;
@@ -200,6 +199,7 @@ notmuch_tag_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
     const char *input_file_name = NULL;
     int opt_index;
     int ret;
+    notmuch_bool_t synchronize_flags;
 
     /* Set up our handler for SIGINT */
     memset (&action, 0, sizeof (struct sigaction));
@@ -240,13 +240,13 @@ notmuch_tag_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 	    return EXIT_FAILURE;
 	}
     } else {
-	tag_ops = tag_op_list_create (config);
+	tag_ops = tag_op_list_create (notmuch);
 	if (tag_ops == NULL) {
 	    fprintf (stderr, "Out of memory.\n");
 	    return EXIT_FAILURE;
 	}
 
-	if (parse_tag_command_line (config, argc - opt_index, argv + opt_index,
+	if (parse_tag_command_line (notmuch, argc - opt_index, argv + opt_index,
 				    &query_string, tag_ops))
 	    return EXIT_FAILURE;
 
@@ -261,22 +261,25 @@ notmuch_tag_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 	}
     }
 
-    if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
-	return EXIT_FAILURE;
-
     notmuch_exit_if_unmatched_db_uuid (notmuch);
 
-    if (notmuch_config_get_maildir_synchronize_flags (config))
+    if (print_status_database (
+	    "notmuch restore",
+	    notmuch,
+	    notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS,
+				     &synchronize_flags)))
+	return EXIT_FAILURE;
+
+    if (synchronize_flags)
 	tag_flags |= TAG_FLAG_MAILDIR_SYNC;
 
     if (remove_all)
 	tag_flags |= TAG_FLAG_REMOVE_ALL;
 
     if (batch)
-	ret = tag_file (config, notmuch, tag_flags, input);
+	ret = tag_file (notmuch, notmuch, tag_flags, input);
     else
-	ret = tag_query (config, notmuch, query_string, tag_ops, tag_flags);
+	ret = tag_query (notmuch, notmuch, query_string, tag_ops, tag_flags);
 
     notmuch_database_destroy (notmuch);
 
diff --git a/notmuch.c b/notmuch.c
index 16504fc5..913fd312 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -157,7 +157,7 @@ static command_t commands[] = {
       "Count messages matching the search terms." },
     { "reply", notmuch_reply_command, NOTMUCH_COMMAND_DATABASE_EARLY,
       "Construct a reply template for a set of messages." },
-    { "tag", notmuch_tag_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "tag", notmuch_tag_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Add/remove tags for all messages matching the search terms." },
     { "dump", notmuch_dump_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Create a plain-text dump of the tags for each message." },
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 23/38] lib/config: add _notmuch_config_cache
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (21 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 22/38] cli/tag: convert " David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 24/38] lib: split notmuch_database_compact David Bremner
                   ` (14 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is a simple convenience routine to cache a configuration value
without writing it to the database.
---
 lib/config.cc         | 5 +++++
 lib/notmuch-private.h | 4 ++++
 2 files changed, 9 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 1318cb52..57eedfb7 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -464,3 +464,8 @@ notmuch_config_set (notmuch_database_t *notmuch, notmuch_config_key_t key, const
 
     return notmuch_database_set_config (notmuch, _notmuch_config_key_to_string (key), val);
 }
+
+void
+_notmuch_config_cache (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val) {
+    _notmuch_string_map_set (notmuch->config, _notmuch_config_key_to_string (key), val);
+}
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 961d50cf..5e0b42e3 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -713,6 +713,10 @@ _notmuch_config_load_from_file (notmuch_database_t * db, GKeyFile *file);
 
 notmuch_status_t
 _notmuch_config_load_defaults (notmuch_database_t * db);
+
+void
+_notmuch_config_cache (notmuch_database_t *db, notmuch_config_key_t key, const char* val);
+
 NOTMUCH_END_DECLS
 
 #ifdef __cplusplus
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 24/38] lib: split notmuch_database_compact
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (22 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 23/38] lib/config: add _notmuch_config_cache David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 25/38] cli/compact: convert to new configuration framework David Bremner
                   ` (13 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The "back end" function takes an open notmuch database, which should
know its own path (i.e. the path needs to be cached in the
configuration data).
---
 lib/database.cc | 42 +++++++++++++++++++++++++++++++++---------
 lib/notmuch.h   | 12 ++++++++++++
 2 files changed, 45 insertions(+), 9 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 0f4e2ff9..650359f4 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -705,27 +705,51 @@ notmuch_database_compact (const char *path,
 			  notmuch_compact_status_cb_t status_cb,
 			  void *closure)
 {
-    void *local;
-    char *notmuch_path, *xapian_path, *compact_xapian_path;
     notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
     notmuch_database_t *notmuch = NULL;
-    struct stat statbuf;
-    bool keep_backup;
     char *message = NULL;
 
-    local = talloc_new (NULL);
-    if (! local)
-	return NOTMUCH_STATUS_OUT_OF_MEMORY;
-
     ret = notmuch_database_open_verbose (path,
 					 NOTMUCH_DATABASE_MODE_READ_WRITE,
 					 &notmuch,
 					 &message);
     if (ret) {
 	if (status_cb) status_cb (message, closure);
-	goto DONE;
+	return ret;
     }
 
+    _notmuch_config_cache (notmuch, NOTMUCH_CONFIG_DATABASE_PATH, path);
+
+    return notmuch_database_compact_db (notmuch,
+					backup_path,
+					status_cb,
+					closure);
+}
+
+notmuch_status_t
+notmuch_database_compact_db (notmuch_database_t *notmuch,
+			     const char *backup_path,
+			     notmuch_compact_status_cb_t status_cb,
+			     void *closure) {
+    void *local;
+    char *notmuch_path, *xapian_path, *compact_xapian_path;
+    const char* path;
+    notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
+    struct stat statbuf;
+    bool keep_backup;
+
+    ret = _notmuch_database_ensure_writable (notmuch);
+    if (ret)
+	return ret;
+
+    path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH);
+    if (! path)
+	return NOTMUCH_STATUS_PATH_ERROR;
+
+    local = talloc_new (NULL);
+    if (! local)
+	return NOTMUCH_STATUS_OUT_OF_MEMORY;
+
     if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) {
 	ret = NOTMUCH_STATUS_OUT_OF_MEMORY;
 	goto DONE;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 4a513b44..4e5312ad 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -497,6 +497,18 @@ notmuch_database_compact (const char *path,
 			  notmuch_compact_status_cb_t status_cb,
 			  void *closure);
 
+/**
+ * Like notmuch_database_compact, but take an open database as a
+ * parameter.
+ *
+ * @since libnnotmuch 5.4 (notmuch 0.32)
+ */
+notmuch_status_t
+notmuch_database_compact_db (notmuch_database_t *database,
+			     const char *backup_path,
+			     notmuch_compact_status_cb_t status_cb,
+			     void *closure);
+
 /**
  * Destroy the notmuch database, closing it if necessary and freeing
  * all associated resources.
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 25/38] cli/compact: convert to new configuration framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (23 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 24/38] lib: split notmuch_database_compact David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 26/38] bindings/notmuch2: add missing crypto error status codes David Bremner
                   ` (12 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Switch to the newly created API function notmuch_database_compact_db,
which takes the database opened in main().
---
 notmuch-compact.c | 7 +++----
 notmuch.c         | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/notmuch-compact.c b/notmuch-compact.c
index ab2066e1..361583db 100644
--- a/notmuch-compact.c
+++ b/notmuch-compact.c
@@ -27,9 +27,8 @@ status_update_cb (const char *msg, unused (void *closure))
 }
 
 int
-notmuch_compact_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_compact_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
-    const char *path = notmuch_config_get_database_path (config);
     const char *backup_path = NULL;
     notmuch_status_t ret;
     bool quiet = false;
@@ -55,8 +54,8 @@ notmuch_compact_command (notmuch_config_t *config, unused(notmuch_database_t *no
 
     if (! quiet)
 	printf ("Compacting database...\n");
-    ret = notmuch_database_compact (path, backup_path,
-				    quiet ? NULL : status_update_cb, NULL);
+    ret = notmuch_database_compact_db (notmuch, backup_path,
+				       quiet ? NULL : status_update_cb, NULL);
     if (ret) {
 	fprintf (stderr, "Compaction failed: %s\n", notmuch_status_to_string (ret));
 	return EXIT_FAILURE;
diff --git a/notmuch.c b/notmuch.c
index 913fd312..3a299afa 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -163,7 +163,7 @@ static command_t commands[] = {
       "Create a plain-text dump of the tags for each message." },
     { "restore", notmuch_restore_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Restore the tags from the given dump file (see 'dump')." },
-    { "compact", notmuch_compact_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "compact", notmuch_compact_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Compact the notmuch database." },
     { "reindex", notmuch_reindex_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Re-index all messages matching the search terms." },
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 26/38] bindings/notmuch2: add missing crypto error status codes
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (24 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 25/38] cli/compact: convert to new configuration framework David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 27/38] lib/config: add NOTMUCH_CONFIG_NEW_IGNORE David Bremner
                   ` (11 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

These are needed so that the later codes line up numerically.
---
 bindings/python-cffi/notmuch2/_build.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py
index f269f2a1..d3bb9104 100644
--- a/bindings/python-cffi/notmuch2/_build.py
+++ b/bindings/python-cffi/notmuch2/_build.py
@@ -47,6 +47,9 @@ ffibuilder.cdef(
         NOTMUCH_STATUS_UPGRADE_REQUIRED,
         NOTMUCH_STATUS_PATH_ERROR,
         NOTMUCH_STATUS_ILLEGAL_ARGUMENT,
+        NOTMUCH_STATUS_MALFORMED_CRYPTO_PROTOCOL,
+        NOTMUCH_STATUS_FAILED_CRYPTO_CONTEXT_CREATION,
+        NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL,
         NOTMUCH_STATUS_LAST_STATUS
     } notmuch_status_t;
     typedef enum {
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 27/38] lib/config: add NOTMUCH_CONFIG_NEW_IGNORE
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (25 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 26/38] bindings/notmuch2: add missing crypto error status codes David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 28/38] lib/open: factor out first part of open David Bremner
                   ` (10 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will be needed by (at least) the conversion of notmuch-new.c to
the new config framework
---
 lib/config.cc          | 3 +++
 lib/notmuch.h          | 1 +
 test/T590-libconfig.sh | 1 +
 3 files changed, 5 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 57eedfb7..443dc3a6 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -394,6 +394,8 @@ _notmuch_config_key_to_string (notmuch_config_key_t key) {
 	return "search.exclude_tags";
     case NOTMUCH_CONFIG_NEW_TAGS:
 	return "new.tags";
+    case NOTMUCH_CONFIG_NEW_IGNORE:
+	return "new.ignore";
     case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS:
 	return "maildir.synchronize_flags";
     case NOTMUCH_CONFIG_PRIMARY_EMAIL:
@@ -426,6 +428,7 @@ _notmuch_config_default (void *ctx, notmuch_config_key_t key) {
 	return "inbox;unread";
     case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS:
 	return "true";
+    case NOTMUCH_CONFIG_NEW_IGNORE:
     case NOTMUCH_CONFIG_USER_NAME:
     case NOTMUCH_CONFIG_PRIMARY_EMAIL:
     case NOTMUCH_CONFIG_OTHER_EMAIL:
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 4e5312ad..5ec14c38 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2427,6 +2427,7 @@ typedef enum _notmuch_config_key {
     NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST,
     NOTMUCH_CONFIG_EXCLUDE_TAGS,
     NOTMUCH_CONFIG_NEW_TAGS,
+    NOTMUCH_CONFIG_NEW_IGNORE,
     NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS,
     NOTMUCH_CONFIG_PRIMARY_EMAIL,
     NOTMUCH_CONFIG_OTHER_EMAIL,
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 5ffe967b..97f8fdc7 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -335,6 +335,7 @@ cat <<'EOF' >EXPECTED
 MAIL_DIR
 
 inbox;unread
+NULL
 true
 NULL
 NULL
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 28/38] lib/open: factor out first part of open
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (26 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 27/38] lib/config: add NOTMUCH_CONFIG_NEW_IGNORE David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 29/38] lib: add NOTMUCH_STATUS_NO_CONFIG David Bremner
                   ` (9 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The plan is to share code with a new database creation function that
has a similar API to n_d_open_with_config.
---
 lib/open.cc | 50 +++++++++++++++++++++++++++++++-------------------
 1 file changed, 31 insertions(+), 19 deletions(-)

diff --git a/lib/open.cc b/lib/open.cc
index 8cede121..07498dec 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -113,6 +113,36 @@ DONE:
     return status;
 }
 
+static notmuch_status_t
+_pre_open(const char *config_path,
+	  const char *profile,
+	  GKeyFile **key_file,
+	  const char **database_path,
+	  char **message)
+{
+    notmuch_status_t status;
+
+    status =_load_key_file (config_path, profile, key_file);
+    if (status) {
+	*message = strdup ("Error: cannot load config file.\n");
+	return status;
+    }
+
+    if (! *database_path && *key_file)
+	*database_path = g_key_file_get_value (*key_file, "database", "path", NULL);
+
+    if (*database_path == NULL) {
+	*message = strdup ("Error: Cannot open a database for a NULL path.\n");
+	return NOTMUCH_STATUS_NULL_POINTER;
+    }
+
+    if (*database_path[0] != '/') {
+	*message = strdup ("Error: Database path must be absolute.\n");
+	return NOTMUCH_STATUS_PATH_ERROR;
+    }
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 notmuch_status_t
 notmuch_database_open_with_config (const char *database_path,
 				   notmuch_database_mode_t mode,
@@ -132,26 +162,8 @@ notmuch_database_open_with_config (const char *database_path,
     GKeyFile *key_file = NULL;
     static int initialized = 0;
 
-    status = _load_key_file (config_path, profile, &key_file);
-    if (status) {
-	message = strdup ("Error: cannot load config file");
-	goto DONE;
-    }
-	
-    if (! database_path && key_file)
-	database_path = g_key_file_get_value (key_file, "database", "path", NULL);
-
-    if (database_path == NULL) {
-	message = strdup ("Error: Cannot open a database for a NULL path.\n");
-	status = NOTMUCH_STATUS_NULL_POINTER;
+    if ((status = _pre_open (config_path, profile, &key_file, &database_path, &message)))
 	goto DONE;
-    }
-
-    if (database_path[0] != '/') {
-	message = strdup ("Error: Database path must be absolute.\n");
-	status = NOTMUCH_STATUS_PATH_ERROR;
-	goto DONE;
-    }
 
     if (! (notmuch_path = talloc_asprintf (local, "%s/%s", database_path, ".notmuch"))) {
 	message = strdup ("Out of memory\n");
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 29/38] lib: add NOTMUCH_STATUS_NO_CONFIG
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (27 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 28/38] lib/open: factor out first part of open David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 30/38] lib/database: move n_d_create* to open.cc David Bremner
                   ` (8 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will allow client code to provide more meaningful diagnostics. In
particular it will enable "notmuch new" to continue suggsting the user
run "notmuch setup" to create a config after "notmuch new" is
transitioned to the new  configuration framework.
---
 bindings/python-cffi/notmuch2/_build.py  | 1 +
 bindings/python-cffi/notmuch2/_errors.py | 4 +++-
 lib/notmuch.h                            | 4 ++++
 lib/open.cc                              | 2 +-
 4 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py
index d3bb9104..93828627 100644
--- a/bindings/python-cffi/notmuch2/_build.py
+++ b/bindings/python-cffi/notmuch2/_build.py
@@ -50,6 +50,7 @@ ffibuilder.cdef(
         NOTMUCH_STATUS_MALFORMED_CRYPTO_PROTOCOL,
         NOTMUCH_STATUS_FAILED_CRYPTO_CONTEXT_CREATION,
         NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL,
+        NOTMUCH_STATUS_NO_CONFIG,
         NOTMUCH_STATUS_LAST_STATUS
     } notmuch_status_t;
     typedef enum {
diff --git a/bindings/python-cffi/notmuch2/_errors.py b/bindings/python-cffi/notmuch2/_errors.py
index 13369445..c97d99cb 100644
--- a/bindings/python-cffi/notmuch2/_errors.py
+++ b/bindings/python-cffi/notmuch2/_errors.py
@@ -50,6 +50,8 @@ class NotmuchError(Exception):
                 PathError,
             capi.lib.NOTMUCH_STATUS_ILLEGAL_ARGUMENT:
                 IllegalArgumentError,
+            capi.lib.NOTMUCH_STATUS_NO_CONFIG:
+                NoConfigError,
         }
         return types[status]
 
@@ -94,7 +96,7 @@ class UnsupportedOperationError(NotmuchError): pass
 class UpgradeRequiredError(NotmuchError): pass
 class PathError(NotmuchError): pass
 class IllegalArgumentError(NotmuchError): pass
-
+class NoConfigError(NotmuchError): pass
 
 class ObjectDestroyedError(NotmuchError):
     """The object has already been destroyed and it's memory freed.
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 5ec14c38..4f384e58 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -208,6 +208,10 @@ typedef enum _notmuch_status {
      * something that notmuch doesn't know how to handle.
      */
     NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL,
+    /**
+     * Unable to load a config file
+     */
+    NOTMUCH_STATUS_NO_CONFIG,
     /**
      * Not an actual status value. Just a way to find out how many
      * valid status values there are.
diff --git a/lib/open.cc b/lib/open.cc
index 07498dec..5103ba98 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -105,7 +105,7 @@ _load_key_file (const char *path,
 
     *key_file = g_key_file_new ();
     if (! g_key_file_load_from_file (*key_file, path, G_KEY_FILE_NONE, NULL)) {
-	status = NOTMUCH_STATUS_FILE_ERROR;
+	status = NOTMUCH_STATUS_NO_CONFIG;
     }
 
 DONE:
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 30/38] lib/database: move n_d_create* to open.cc
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (28 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 29/38] lib: add NOTMUCH_STATUS_NO_CONFIG David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:03 ` [PATCH 31/38] lib: add NOTMUCH_STATUS_DATABASE_EXISTS David Bremner
                   ` (7 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will help share code with n_d_open_with_config.
---
 lib/database.cc | 103 ------------------------------------------------
 lib/open.cc     | 103 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 103 insertions(+), 103 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 650359f4..f96ba7c0 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -453,109 +453,6 @@ notmuch_database_find_message (notmuch_database_t *notmuch,
     }
 }
 
-notmuch_status_t
-notmuch_database_create (const char *path, notmuch_database_t **database)
-{
-    char *status_string = NULL;
-    notmuch_status_t status;
-
-    status = notmuch_database_create_verbose (path, database,
-					      &status_string);
-
-    if (status_string) {
-	fputs (status_string, stderr);
-	free (status_string);
-    }
-
-    return status;
-}
-
-notmuch_status_t
-notmuch_database_create_verbose (const char *path,
-				 notmuch_database_t **database,
-				 char **status_string)
-{
-    notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
-    notmuch_database_t *notmuch = NULL;
-    char *notmuch_path = NULL;
-    char *message = NULL;
-    struct stat st;
-    int err;
-
-    if (path == NULL) {
-	message = strdup ("Error: Cannot create a database for a NULL path.\n");
-	status = NOTMUCH_STATUS_NULL_POINTER;
-	goto DONE;
-    }
-
-    if (path[0] != '/') {
-	message = strdup ("Error: Database path must be absolute.\n");
-	status = NOTMUCH_STATUS_PATH_ERROR;
-	goto DONE;
-    }
-
-    err = stat (path, &st);
-    if (err) {
-	IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: %s.\n",
-				 path, strerror (errno)));
-	status = NOTMUCH_STATUS_FILE_ERROR;
-	goto DONE;
-    }
-
-    if (! S_ISDIR (st.st_mode)) {
-	IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: "
-				 "Not a directory.\n",
-				 path));
-	status = NOTMUCH_STATUS_FILE_ERROR;
-	goto DONE;
-    }
-
-    notmuch_path = talloc_asprintf (NULL, "%s/%s", path, ".notmuch");
-
-    err = mkdir (notmuch_path, 0755);
-
-    if (err) {
-	IGNORE_RESULT (asprintf (&message, "Error: Cannot create directory %s: %s.\n",
-				 notmuch_path, strerror (errno)));
-	status = NOTMUCH_STATUS_FILE_ERROR;
-	goto DONE;
-    }
-
-    status = notmuch_database_open_verbose (path,
-					    NOTMUCH_DATABASE_MODE_READ_WRITE,
-					    &notmuch, &message);
-    if (status)
-	goto DONE;
-
-    /* Upgrade doesn't add these feature to existing databases, but
-     * new databases have them. */
-    notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
-    notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;
-    notmuch->features |= NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY;
-
-    status = notmuch_database_upgrade (notmuch, NULL, NULL);
-    if (status) {
-	notmuch_database_close (notmuch);
-	notmuch = NULL;
-    }
-
-  DONE:
-    if (notmuch_path)
-	talloc_free (notmuch_path);
-
-    if (message) {
-	if (status_string)
-	    *status_string = message;
-	else
-	    free (message);
-    }
-    if (database)
-	*database = notmuch;
-    else
-	talloc_free (notmuch);
-    return status;
-}
-
 notmuch_status_t
 _notmuch_database_ensure_writable (notmuch_database_t *notmuch)
 {
diff --git a/lib/open.cc b/lib/open.cc
index 5103ba98..0f392ab9 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -340,3 +340,106 @@ notmuch_database_open_with_config (const char *database_path,
 
     return status;
 }
+
+notmuch_status_t
+notmuch_database_create (const char *path, notmuch_database_t **database)
+{
+    char *status_string = NULL;
+    notmuch_status_t status;
+
+    status = notmuch_database_create_verbose (path, database,
+					      &status_string);
+
+    if (status_string) {
+	fputs (status_string, stderr);
+	free (status_string);
+    }
+
+    return status;
+}
+
+notmuch_status_t
+notmuch_database_create_verbose (const char *path,
+				 notmuch_database_t **database,
+				 char **status_string)
+{
+    notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+    notmuch_database_t *notmuch = NULL;
+    char *notmuch_path = NULL;
+    char *message = NULL;
+    struct stat st;
+    int err;
+
+    if (path == NULL) {
+	message = strdup ("Error: Cannot create a database for a NULL path.\n");
+	status = NOTMUCH_STATUS_NULL_POINTER;
+	goto DONE;
+    }
+
+    if (path[0] != '/') {
+	message = strdup ("Error: Database path must be absolute.\n");
+	status = NOTMUCH_STATUS_PATH_ERROR;
+	goto DONE;
+    }
+
+    err = stat (path, &st);
+    if (err) {
+	IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: %s.\n",
+				 path, strerror (errno)));
+	status = NOTMUCH_STATUS_FILE_ERROR;
+	goto DONE;
+    }
+
+    if (! S_ISDIR (st.st_mode)) {
+	IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: "
+				 "Not a directory.\n",
+				 path));
+	status = NOTMUCH_STATUS_FILE_ERROR;
+	goto DONE;
+    }
+
+    notmuch_path = talloc_asprintf (NULL, "%s/%s", path, ".notmuch");
+
+    err = mkdir (notmuch_path, 0755);
+
+    if (err) {
+	IGNORE_RESULT (asprintf (&message, "Error: Cannot create directory %s: %s.\n",
+				 notmuch_path, strerror (errno)));
+	status = NOTMUCH_STATUS_FILE_ERROR;
+	goto DONE;
+    }
+
+    status = notmuch_database_open_verbose (path,
+					    NOTMUCH_DATABASE_MODE_READ_WRITE,
+					    &notmuch, &message);
+    if (status)
+	goto DONE;
+
+    /* Upgrade doesn't add these feature to existing databases, but
+     * new databases have them. */
+    notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+    notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;
+    notmuch->features |= NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY;
+
+    status = notmuch_database_upgrade (notmuch, NULL, NULL);
+    if (status) {
+	notmuch_database_close (notmuch);
+	notmuch = NULL;
+    }
+
+  DONE:
+    if (notmuch_path)
+	talloc_free (notmuch_path);
+
+    if (message) {
+	if (status_string)
+	    *status_string = message;
+	else
+	    free (message);
+    }
+    if (database)
+	*database = notmuch;
+    else
+	talloc_free (notmuch);
+    return status;
+}
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 31/38] lib: add NOTMUCH_STATUS_DATABASE_EXISTS
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (29 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 30/38] lib/database: move n_d_create* to open.cc David Bremner
@ 2021-01-16 17:03 ` David Bremner
  2021-01-16 17:04 ` [PATCH 32/38] lib: introduce notmuch_database_create_with_config David Bremner
                   ` (6 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:03 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

It is desirable to distinguish between attempting to create a database
that already exists, and more fatal errors like permission problems.
---
 bindings/python-cffi/notmuch2/_build.py  | 1 +
 bindings/python-cffi/notmuch2/_errors.py | 3 +++
 lib/notmuch.h                            | 4 ++++
 3 files changed, 8 insertions(+)

diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py
index 93828627..f67b4de6 100644
--- a/bindings/python-cffi/notmuch2/_build.py
+++ b/bindings/python-cffi/notmuch2/_build.py
@@ -51,6 +51,7 @@ ffibuilder.cdef(
         NOTMUCH_STATUS_FAILED_CRYPTO_CONTEXT_CREATION,
         NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL,
         NOTMUCH_STATUS_NO_CONFIG,
+        NOTMUCH_STATUS_DATABASE_EXISTS,
         NOTMUCH_STATUS_LAST_STATUS
     } notmuch_status_t;
     typedef enum {
diff --git a/bindings/python-cffi/notmuch2/_errors.py b/bindings/python-cffi/notmuch2/_errors.py
index c97d99cb..65064d4e 100644
--- a/bindings/python-cffi/notmuch2/_errors.py
+++ b/bindings/python-cffi/notmuch2/_errors.py
@@ -52,6 +52,8 @@ class NotmuchError(Exception):
                 IllegalArgumentError,
             capi.lib.NOTMUCH_STATUS_NO_CONFIG:
                 NoConfigError,
+            capi.lib.NOTMUCH_STATUS_DATABASE_EXISTS:
+                DatabaseExistsError,
         }
         return types[status]
 
@@ -97,6 +99,7 @@ class UpgradeRequiredError(NotmuchError): pass
 class PathError(NotmuchError): pass
 class IllegalArgumentError(NotmuchError): pass
 class NoConfigError(NotmuchError): pass
+class DatabaseExistsError(NotmuchError): pass
 
 class ObjectDestroyedError(NotmuchError):
     """The object has already been destroyed and it's memory freed.
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 4f384e58..6d22d328 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -212,6 +212,10 @@ typedef enum _notmuch_status {
      * Unable to load a config file
      */
     NOTMUCH_STATUS_NO_CONFIG,
+    /**
+     * Database exists, so not (re)-created
+     */
+    NOTMUCH_STATUS_DATABASE_EXISTS,
     /**
      * Not an actual status value. Just a way to find out how many
      * valid status values there are.
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 32/38] lib: introduce notmuch_database_create_with_config
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (30 preceding siblings ...)
  2021-01-16 17:03 ` [PATCH 31/38] lib: add NOTMUCH_STATUS_DATABASE_EXISTS David Bremner
@ 2021-01-16 17:04 ` David Bremner
  2021-01-16 17:04 ` [PATCH 33/38] cli/new: refactor database upgrade code David Bremner
                   ` (5 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This takes a config path parameter, and can use that to decide the
new database location.
---
 bindings/python-cffi/tests/test_database.py |  2 +-
 lib/notmuch.h                               | 28 ++++++++++++
 lib/open.cc                                 | 50 ++++++++++++---------
 test/T560-lib-error.sh                      |  2 +-
 4 files changed, 59 insertions(+), 23 deletions(-)

diff --git a/bindings/python-cffi/tests/test_database.py b/bindings/python-cffi/tests/test_database.py
index a2c69de6..9b3219c0 100644
--- a/bindings/python-cffi/tests/test_database.py
+++ b/bindings/python-cffi/tests/test_database.py
@@ -80,7 +80,7 @@ class TestCreate:
             db.create(tmppath)
 
     def test_create_existing(self, tmppath, db):
-        with pytest.raises(errors.FileError):
+        with pytest.raises(errors.DatabaseExistsError):
             dbmod.Database.create(path=tmppath)
 
     def test_close(self, db):
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 6d22d328..0f14b569 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -442,6 +442,34 @@ notmuch_database_open_with_config (const char *database_path,
 				   const char *profile,
 				   notmuch_database_t **database,
 				   char **error_message);
+/**
+ * Create a new notmuch database located at 'database_path', using
+ * configuration in 'config_path'.
+ *
+ * @param[in,out] is_new if NULL consider an existing database as an
+ *		  error.  Otherwise report whether actually created.
+ *
+ * For description of other arguments, see notmuch_database_open_with_config
+ *
+ * @retval NOTMUCH_STATUS_SUCCESS: Successfully created the database.
+ *
+ * @retval NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory.
+ *
+ * @retval NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the
+ *	database or config file (such as permission denied, or file not found,
+ *	etc.)
+ *
+ * @retval NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred.
+ *
+ * @since libnotmuch 5.4 (notmuch 0.32)
+ */
+
+notmuch_status_t
+notmuch_database_create_with_config (const char *database_path,
+				     const char *config_path,
+				     const char *profile,
+				     notmuch_database_t **database,
+				     char **error_message);
 
 /**
  * Retrieve last status string for given database.
diff --git a/lib/open.cc b/lib/open.cc
index 0f392ab9..92661271 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -362,30 +362,32 @@ notmuch_status_t
 notmuch_database_create_verbose (const char *path,
 				 notmuch_database_t **database,
 				 char **status_string)
+{
+    return notmuch_database_create_with_config (path, "", NULL, database, status_string);
+}
+
+notmuch_status_t
+notmuch_database_create_with_config (const char *database_path,
+				     const char *config_path,
+				     const char *profile,
+				     notmuch_database_t **database,
+				     char **status_string)
 {
     notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
     notmuch_database_t *notmuch = NULL;
     char *notmuch_path = NULL;
     char *message = NULL;
+    GKeyFile *key_file = NULL;
     struct stat st;
     int err;
 
-    if (path == NULL) {
-	message = strdup ("Error: Cannot create a database for a NULL path.\n");
-	status = NOTMUCH_STATUS_NULL_POINTER;
-	goto DONE;
-    }
-
-    if (path[0] != '/') {
-	message = strdup ("Error: Database path must be absolute.\n");
-	status = NOTMUCH_STATUS_PATH_ERROR;
+    if ((status = _pre_open (config_path, profile, &key_file, &database_path, &message)))
 	goto DONE;
-    }
 
-    err = stat (path, &st);
+    err = stat (database_path, &st);
     if (err) {
 	IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: %s.\n",
-				 path, strerror (errno)));
+				 database_path, strerror (errno)));
 	status = NOTMUCH_STATUS_FILE_ERROR;
 	goto DONE;
     }
@@ -393,25 +395,31 @@ notmuch_database_create_verbose (const char *path,
     if (! S_ISDIR (st.st_mode)) {
 	IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: "
 				 "Not a directory.\n",
-				 path));
+				 database_path));
 	status = NOTMUCH_STATUS_FILE_ERROR;
 	goto DONE;
     }
 
-    notmuch_path = talloc_asprintf (NULL, "%s/%s", path, ".notmuch");
+    notmuch_path = talloc_asprintf (NULL, "%s/%s", database_path, ".notmuch");
 
     err = mkdir (notmuch_path, 0755);
-
     if (err) {
-	IGNORE_RESULT (asprintf (&message, "Error: Cannot create directory %s: %s.\n",
-				 notmuch_path, strerror (errno)));
-	status = NOTMUCH_STATUS_FILE_ERROR;
+	if (errno == EEXIST) {
+	    status = NOTMUCH_STATUS_DATABASE_EXISTS;
+	} else {
+	    IGNORE_RESULT (asprintf (&message, "Error: Cannot create directory %s: %s.\n",
+				     notmuch_path, strerror (errno)));
+	    status = NOTMUCH_STATUS_FILE_ERROR;
+	}
 	goto DONE;
     }
 
-    status = notmuch_database_open_verbose (path,
-					    NOTMUCH_DATABASE_MODE_READ_WRITE,
-					    &notmuch, &message);
+    /* XXX this reads the config file twice, which is a bit wasteful */
+    status = notmuch_database_open_with_config (database_path,
+						NOTMUCH_DATABASE_MODE_READ_WRITE,
+						config_path,
+						profile,
+						&notmuch, &message);
     if (status)
 	goto DONE;
 
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 260ac120..ade376ef 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -93,7 +93,7 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 == stderr ==
-Error: Cannot create a database for a NULL path.
+Error: Cannot open a database for a NULL path.
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 33/38] cli/new: refactor database upgrade code
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (31 preceding siblings ...)
  2021-01-16 17:04 ` [PATCH 32/38] lib: introduce notmuch_database_create_with_config David Bremner
@ 2021-01-16 17:04 ` David Bremner
  2021-01-16 17:04 ` [PATCH 34/38] cli/new: convert to new config framework David Bremner
                   ` (4 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Move to a separate function. This is essentially just code movement.
---
 notmuch-new.c | 97 ++++++++++++++++++++++++++++-----------------------
 1 file changed, 53 insertions(+), 44 deletions(-)

diff --git a/notmuch-new.c b/notmuch-new.c
index e0e3de25..a5c3cb54 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -1042,6 +1042,57 @@ print_results (const add_files_state_t *state)
     printf ("\n");
 }
 
+static int
+_maybe_upgrade (notmuch_database_t *notmuch, add_files_state_t *state) {
+    if (notmuch_database_needs_upgrade (notmuch)) {
+	time_t now = time (NULL);
+	struct tm *gm_time = gmtime (&now);
+	notmuch_status_t status;
+	char *dot_notmuch_path = talloc_asprintf (notmuch, "%s/%s", state->db_path, ".notmuch");
+
+	/* since dump files are written atomically, the amount of
+	 * harm from overwriting one within a second seems
+	 * relatively small. */
+
+	const char *backup_name =
+	    talloc_asprintf (notmuch, "%s/dump-%04d%02d%02dT%02d%02d%02d.gz",
+			     dot_notmuch_path,
+			     gm_time->tm_year + 1900,
+			     gm_time->tm_mon + 1,
+			     gm_time->tm_mday,
+			     gm_time->tm_hour,
+			     gm_time->tm_min,
+			     gm_time->tm_sec);
+
+	if (state->verbosity >= VERBOSITY_NORMAL) {
+	    printf ("Welcome to a new version of notmuch! Your database will now be upgraded.\n");
+	    printf ("This process is safe to interrupt.\n");
+	    printf ("Backing up tags to %s...\n", backup_name);
+	}
+
+	if (notmuch_database_dump (notmuch, backup_name, "",
+				   DUMP_FORMAT_BATCH_TAG, DUMP_INCLUDE_DEFAULT, true)) {
+	    fprintf (stderr, "Backup failed. Aborting upgrade.");
+	    return EXIT_FAILURE;
+	}
+
+	gettimeofday (&state->tv_start, NULL);
+	status = notmuch_database_upgrade (
+	    notmuch,
+	    state->verbosity >= VERBOSITY_NORMAL ? upgrade_print_progress : NULL,
+	    state);
+	if (status) {
+	    printf ("Upgrade failed: %s\n",
+		    notmuch_status_to_string (status));
+	    notmuch_database_destroy (notmuch);
+	    return EXIT_FAILURE;
+	}
+	if (state->verbosity >= VERBOSITY_NORMAL)
+	    printf ("Your notmuch database has now been upgraded.\n");
+    }
+    return EXIT_SUCCESS;
+}
+
 int
 notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
 {
@@ -1142,50 +1193,8 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 
 	notmuch_exit_if_unmatched_db_uuid (notmuch);
 
-	if (notmuch_database_needs_upgrade (notmuch)) {
-	    time_t now = time (NULL);
-	    struct tm *gm_time = gmtime (&now);
-
-	    /* since dump files are written atomically, the amount of
-	     * harm from overwriting one within a second seems
-	     * relatively small. */
-
-	    const char *backup_name =
-		talloc_asprintf (notmuch, "%s/dump-%04d%02d%02dT%02d%02d%02d.gz",
-				 dot_notmuch_path,
-				 gm_time->tm_year + 1900,
-				 gm_time->tm_mon + 1,
-				 gm_time->tm_mday,
-				 gm_time->tm_hour,
-				 gm_time->tm_min,
-				 gm_time->tm_sec);
-
-	    if (add_files_state.verbosity >= VERBOSITY_NORMAL) {
-		printf ("Welcome to a new version of notmuch! Your database will now be upgraded.\n");
-		printf ("This process is safe to interrupt.\n");
-		printf ("Backing up tags to %s...\n", backup_name);
-	    }
-
-	    if (notmuch_database_dump (notmuch, backup_name, "",
-				       DUMP_FORMAT_BATCH_TAG, DUMP_INCLUDE_DEFAULT, true)) {
-		fprintf (stderr, "Backup failed. Aborting upgrade.");
-		return EXIT_FAILURE;
-	    }
-
-	    gettimeofday (&add_files_state.tv_start, NULL);
-	    status = notmuch_database_upgrade (
-		notmuch,
-		add_files_state.verbosity >= VERBOSITY_NORMAL ? upgrade_print_progress : NULL,
-		&add_files_state);
-	    if (status) {
-		printf ("Upgrade failed: %s\n",
-			notmuch_status_to_string (status));
-		notmuch_database_destroy (notmuch);
-		return EXIT_FAILURE;
-	    }
-	    if (add_files_state.verbosity >= VERBOSITY_NORMAL)
-		printf ("Your notmuch database has now been upgraded.\n");
-	}
+	if (_maybe_upgrade (notmuch, &add_files_state))
+	    return EXIT_FAILURE;
 
 	add_files_state.total_files = 0;
     }
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 34/38] cli/new: convert to new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (32 preceding siblings ...)
  2021-01-16 17:04 ` [PATCH 33/38] cli/new: refactor database upgrade code David Bremner
@ 2021-01-16 17:04 ` David Bremner
  2021-01-16 17:04 ` [PATCH 35/38] lib/config: add HOOK_DIR David Bremner
                   ` (3 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

In addition to the same type of changes as converting other
subcommands, add the possibility of creating a database at the top
level. It would probably make sense to use this for insert as well.
---
 notmuch-client.h   |   1 +
 notmuch-new.c      | 112 ++++++++++++++++++++++-----------------------
 notmuch.c          |  50 +++++++++++++++-----
 test/T040-setup.sh |   2 +-
 test/T050-new.sh   |   6 +--
 5 files changed, 97 insertions(+), 74 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index a026002a..9e09c36a 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -257,6 +257,7 @@ typedef enum {
     NOTMUCH_COMMAND_CONFIG_CREATE	= 1 << 1,
     NOTMUCH_COMMAND_DATABASE_EARLY	= 1 << 2,
     NOTMUCH_COMMAND_DATABASE_WRITE	= 1 << 3,
+    NOTMUCH_COMMAND_DATABASE_CREATE	= 1 << 4,
 } notmuch_command_mode_t;
 
 notmuch_config_t *
diff --git a/notmuch-new.c b/notmuch-new.c
index a5c3cb54..0f416939 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -48,8 +48,7 @@ typedef struct {
     enum verbosity verbosity;
     bool debug;
     bool full_scan;
-    const char **new_tags;
-    size_t new_tags_length;
+    notmuch_config_values_t *new_tags;
     const char **ignore_verbatim;
     size_t ignore_verbatim_length;
     regex_t *ignore_regex;
@@ -65,7 +64,7 @@ typedef struct {
     _filename_list_t *removed_directories;
     _filename_list_t *directory_mtimes;
 
-    bool synchronize_flags;
+    notmuch_bool_t synchronize_flags;
 } add_files_state_t;
 
 static volatile sig_atomic_t do_print_progress = 0;
@@ -245,19 +244,17 @@ _special_directory (const char *entry)
 }
 
 static bool
-_setup_ignore (notmuch_config_t *config, add_files_state_t *state)
+_setup_ignore (notmuch_database_t *notmuch, add_files_state_t *state)
 {
-    const char **ignore_list, **ignore;
+    notmuch_config_values_t *ignore_list;
     int nregex = 0, nverbatim = 0;
     const char **verbatim = NULL;
     regex_t *regex = NULL;
 
-    ignore_list = notmuch_config_get_new_ignore (config, NULL);
-    if (! ignore_list)
-	return true;
-
-    for (ignore = ignore_list; *ignore; ignore++) {
-	const char *s = *ignore;
+    for (ignore_list = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_IGNORE);
+	 notmuch_config_values_valid (ignore_list);
+	 notmuch_config_values_move_to_next (ignore_list)) {
+	const char *s = notmuch_config_values_get (ignore_list);
 	size_t len = strlen (s);
 
 	if (len == 0) {
@@ -276,8 +273,8 @@ _setup_ignore (notmuch_config_t *config, add_files_state_t *state)
 		return false;
 	    }
 
-	    r = talloc_strndup (config, s + 1, len - 2);
-	    regex = talloc_realloc (config, regex, regex_t, nregex + 1);
+	    r = talloc_strndup (notmuch, s + 1, len - 2);
+	    regex = talloc_realloc (notmuch, regex, regex_t, nregex + 1);
 	    preg = &regex[nregex];
 
 	    rerr = regcomp (preg, r, REG_EXTENDED | REG_NOSUB);
@@ -295,7 +292,7 @@ _setup_ignore (notmuch_config_t *config, add_files_state_t *state)
 
 	    talloc_free (r);
 	} else {
-	    verbatim = talloc_realloc (config, verbatim, const char *,
+	    verbatim = talloc_realloc (notmuch, verbatim, const char *,
 				       nverbatim + 1);
 	    verbatim[nverbatim++] = s;
 	}
@@ -371,7 +368,7 @@ add_file (notmuch_database_t *notmuch, const char *filename,
 	  add_files_state_t *state)
 {
     notmuch_message_t *message = NULL;
-    const char **tag;
+    const char *tag;
     notmuch_status_t status;
 
     status = notmuch_database_begin_atomic (notmuch);
@@ -387,13 +384,17 @@ add_file (notmuch_database_t *notmuch, const char *filename,
 	if (state->synchronize_flags)
 	    notmuch_message_maildir_flags_to_tags (message);
 
-	for (tag = state->new_tags; *tag != NULL; tag++) {
+	for (notmuch_config_values_start (state->new_tags);
+	     notmuch_config_values_valid (state->new_tags);
+	     notmuch_config_values_move_to_next (state->new_tags)) {
 	    notmuch_bool_t is_set;
+
+	    tag = notmuch_config_values_get (state->new_tags);
 	    /* Currently all errors from has_maildir_flag are fatal */
 	    if ((status = notmuch_message_has_maildir_flag_st (message, 'S', &is_set)))
 		goto DONE;
-	    if (strcmp ("unread", *tag) != 0 || ! is_set) {
-		notmuch_message_add_tag (message, *tag);
+	    if (strcmp ("unread", tag) != 0 || ! is_set) {
+		notmuch_message_add_tag (message, tag);
 	    }
 	}
 
@@ -962,8 +963,7 @@ remove_filename (notmuch_database_t *notmuch,
 /* Recursively remove all filenames from the database referring to
  * 'path' (or to any of its children). */
 static notmuch_status_t
-_remove_directory (void *ctx,
-		   notmuch_database_t *notmuch,
+_remove_directory (notmuch_database_t *notmuch,
 		   const char *path,
 		   add_files_state_t *add_files_state)
 {
@@ -979,7 +979,7 @@ _remove_directory (void *ctx,
     for (files = notmuch_directory_get_child_files (directory);
 	 notmuch_filenames_valid (files);
 	 notmuch_filenames_move_to_next (files)) {
-	absolute = talloc_asprintf (ctx, "%s/%s", path,
+	absolute = talloc_asprintf (notmuch, "%s/%s", path,
 				    notmuch_filenames_get (files));
 	status = remove_filename (notmuch, absolute, add_files_state);
 	talloc_free (absolute);
@@ -990,9 +990,9 @@ _remove_directory (void *ctx,
     for (subdirs = notmuch_directory_get_child_directories (directory);
 	 notmuch_filenames_valid (subdirs);
 	 notmuch_filenames_move_to_next (subdirs)) {
-	absolute = talloc_asprintf (ctx, "%s/%s", path,
+	absolute = talloc_asprintf (notmuch, "%s/%s", path,
 				    notmuch_filenames_get (subdirs));
-	status = _remove_directory (ctx, notmuch, absolute, add_files_state);
+	status = _remove_directory (notmuch, absolute, add_files_state);
 	talloc_free (absolute);
 	if (status)
 	    goto DONE;
@@ -1094,9 +1094,8 @@ _maybe_upgrade (notmuch_database_t *notmuch, add_files_state_t *state) {
 }
 
 int
-notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuch), int argc, char *argv[])
+notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmuch, int argc, char *argv[])
 {
-    notmuch_database_t *notmuch;
     add_files_state_t add_files_state = {
 	.verbosity = VERBOSITY_NORMAL,
 	.debug = false,
@@ -1105,7 +1104,6 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
     };
     struct timeval tv_start;
     int ret = 0;
-    struct stat st;
     const char *db_path;
     char *dot_notmuch_path;
     struct sigaction action;
@@ -1140,21 +1138,30 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
     else if (verbose)
 	add_files_state.verbosity = VERBOSITY_VERBOSE;
 
-    add_files_state.new_tags = notmuch_config_get_new_tags (config, &add_files_state.new_tags_length);
-    add_files_state.synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config);
-    db_path = notmuch_config_get_database_path (config);
+    add_files_state.new_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_TAGS);
+
+    if (print_status_database (
+	    "notmuch new",
+	    notmuch,
+	    notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS,
+				     &add_files_state.synchronize_flags)))
+	return EXIT_FAILURE;
+
+    db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH);
     add_files_state.db_path = db_path;
 
-    if (! _setup_ignore (config, &add_files_state))
+    if (! _setup_ignore (notmuch, &add_files_state))
 	return EXIT_FAILURE;
 
-    for (i = 0; i < add_files_state.new_tags_length; i++) {
-	const char *error_msg;
+    for (notmuch_config_values_start (add_files_state.new_tags);
+	 notmuch_config_values_valid (add_files_state.new_tags);
+	 notmuch_config_values_move_to_next (add_files_state.new_tags)) {
+	const char *tag,*error_msg;
 
-	error_msg = illegal_tag (add_files_state.new_tags[i], false);
+	tag = notmuch_config_values_get (add_files_state.new_tags);
+	error_msg = illegal_tag (tag, false);
 	if (error_msg) {
-	    fprintf (stderr, "Error: tag '%s' in new.tags: %s\n",
-		     add_files_state.new_tags[i], error_msg);
+	    fprintf (stderr, "Error: tag '%s' in new.tags: %s\n", tag, error_msg);
 	    return EXIT_FAILURE;
 	}
     }
@@ -1165,34 +1172,21 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 	    return EXIT_FAILURE;
     }
 
-    dot_notmuch_path = talloc_asprintf (config, "%s/%s", db_path, ".notmuch");
+    dot_notmuch_path = talloc_asprintf (notmuch, "%s/%s", db_path, ".notmuch");
 
-    if (stat (dot_notmuch_path, &st)) {
-	int count;
+    notmuch_exit_if_unmatched_db_uuid (notmuch);
 
-	count = 0;
+    if (notmuch_database_get_revision (notmuch, NULL) == 0) {
+	int count = 0;
 	count_files (db_path, &count, &add_files_state);
 	if (interrupted)
 	    return EXIT_FAILURE;
 
 	if (add_files_state.verbosity >= VERBOSITY_NORMAL)
 	    printf ("Found %d total files (that's not much mail).\n", count);
-	if (notmuch_database_create (db_path, &notmuch))
-	    return EXIT_FAILURE;
+
 	add_files_state.total_files = count;
     } else {
-	char *status_string = NULL;
-	if (notmuch_database_open_verbose (db_path, NOTMUCH_DATABASE_MODE_READ_WRITE,
-					   &notmuch, &status_string)) {
-	    if (status_string) {
-		fputs (status_string, stderr);
-		free (status_string);
-	    }
-	    return EXIT_FAILURE;
-	}
-
-	notmuch_exit_if_unmatched_db_uuid (notmuch);
-
 	if (_maybe_upgrade (notmuch, &add_files_state))
 	    return EXIT_FAILURE;
 
@@ -1223,9 +1217,9 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 
     gettimeofday (&add_files_state.tv_start, NULL);
 
-    add_files_state.removed_files = _filename_list_create (config);
-    add_files_state.removed_directories = _filename_list_create (config);
-    add_files_state.directory_mtimes = _filename_list_create (config);
+    add_files_state.removed_files = _filename_list_create (notmuch);
+    add_files_state.removed_directories = _filename_list_create (notmuch);
+    add_files_state.directory_mtimes = _filename_list_create (notmuch);
 
     if (add_files_state.verbosity == VERBOSITY_NORMAL &&
 	add_files_state.output_is_a_tty && ! debugger_is_active ()) {
@@ -1252,7 +1246,7 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 
     gettimeofday (&tv_start, NULL);
     for (f = add_files_state.removed_directories->head, i = 0; f && ! interrupted; f = f->next, i++) {
-	ret = _remove_directory (config, notmuch, f->filename, &add_files_state);
+	ret = _remove_directory (notmuch, f->filename, &add_files_state);
 	if (ret)
 	    goto DONE;
 	if (do_print_progress) {
@@ -1287,11 +1281,13 @@ notmuch_new_command (notmuch_config_t *config, unused(notmuch_database_t *notmuc
 	fprintf (stderr, "Note: A fatal error was encountered: %s\n",
 		 notmuch_status_to_string (ret));
 
-    notmuch_database_destroy (notmuch);
+    notmuch_database_close (notmuch);
 
     if (hooks && ! ret && ! interrupted)
 	ret = notmuch_run_hook (db_path, "post-new");
 
+    notmuch_database_destroy (notmuch);
+
     if (ret || interrupted)
 	return EXIT_FAILURE;
 
diff --git a/notmuch.c b/notmuch.c
index 3a299afa..7360e0e6 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -143,7 +143,8 @@ static command_t commands[] = {
       "Notmuch main command." },
     { "setup", notmuch_setup_command, NOTMUCH_COMMAND_CONFIG_OPEN | NOTMUCH_COMMAND_CONFIG_CREATE,
       "Interactively set up notmuch for first use." },
-    { "new", notmuch_new_command, NOTMUCH_COMMAND_CONFIG_OPEN,
+    { "new", notmuch_new_command,
+      NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE | NOTMUCH_COMMAND_DATABASE_CREATE,
       "Find and import new messages to the notmuch database." },
     { "insert", notmuch_insert_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE,
       "Add a new message into the maildir and notmuch database." },
@@ -504,23 +505,48 @@ main (int argc, char *argv[])
     if (command->mode & NOTMUCH_COMMAND_DATABASE_EARLY) {
 	char *status_string = NULL;
 	notmuch_database_mode_t mode;
-	if (command->mode & NOTMUCH_COMMAND_DATABASE_WRITE)
+	notmuch_status_t status;
+
+	if (command->mode & NOTMUCH_COMMAND_DATABASE_WRITE ||
+	    command->mode & NOTMUCH_COMMAND_DATABASE_CREATE)
 	    mode = NOTMUCH_DATABASE_MODE_READ_WRITE;
 	else
 	    mode = NOTMUCH_DATABASE_MODE_READ_ONLY;
 
-	if (notmuch_database_open_with_config (NULL,
-					       mode,
-					       config_file_name,
-					       NULL,
-					       &notmuch,
-					       &status_string)) {
-	    if (status_string) {
-		fputs (status_string, stderr);
-		free (status_string);
+	if (command->mode & NOTMUCH_COMMAND_DATABASE_CREATE) {
+	    status = notmuch_database_create_with_config (NULL,
+							  config_file_name,
+							  NULL,
+							  &notmuch,
+							  &status_string);
+	    if (status && status != NOTMUCH_STATUS_DATABASE_EXISTS) {
+		if (status_string) {
+		    fputs (status_string, stderr);
+		    free (status_string);
+		}
+
+		if (status == NOTMUCH_STATUS_NO_CONFIG)
+		    fputs ("Try running 'notmuch setup' to create a configuration.", stderr);
+
+		return EXIT_FAILURE;
 	    }
+	}
 
-	    return EXIT_FAILURE;
+	if (notmuch == NULL) {
+	    status = notmuch_database_open_with_config (NULL,
+							mode,
+							config_file_name,
+							NULL,
+							&notmuch,
+							&status_string);
+	    if (status) {
+		if (status_string) {
+		    fputs (status_string, stderr);
+		    free (status_string);
+		}
+
+		return EXIT_FAILURE;
+	    }
 	}
     } else {
 	config = notmuch_config_open (local, config_file_name, command->mode);
diff --git a/test/T040-setup.sh b/test/T040-setup.sh
index fbfe200a..1223ebf7 100755
--- a/test/T040-setup.sh
+++ b/test/T040-setup.sh
@@ -6,7 +6,7 @@ test_description='"notmuch setup"'
 test_begin_subtest "Notmuch new without a config suggests notmuch setup"
 output=$(notmuch --config=new-notmuch-config new 2>&1)
 test_expect_equal "$output" "\
-Configuration file new-notmuch-config not found.
+Error: cannot load config file.
 Try running 'notmuch setup' to create a configuration."
 
 test_begin_subtest "Create a new config interactively"
diff --git a/test/T050-new.sh b/test/T050-new.sh
index 009b2633..882939fa 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -324,10 +324,10 @@ test_expect_equal "$output" ""
 
 OLDCONFIG=$(notmuch config get new.tags)
 
-test_begin_subtest "Empty tags in new.tags are forbidden"
+test_begin_subtest "Empty tags in new.tags are ignored"
 notmuch config set new.tags "foo;;bar"
-output=$(NOTMUCH_NEW --debug 2>&1)
-test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden"
+output=$(NOTMUCH_NEW --quiet 2>&1)
+test_expect_equal "$output" ""
 
 test_begin_subtest "Tags starting with '-' in new.tags are forbidden"
 notmuch config set new.tags "-foo;bar"
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 35/38] lib/config: add HOOK_DIR
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (33 preceding siblings ...)
  2021-01-16 17:04 ` [PATCH 34/38] cli/new: convert to new config framework David Bremner
@ 2021-01-16 17:04 ` David Bremner
  2021-01-16 17:04 ` [PATCH 36/38] lib/open: set HOOK_DIR on open David Bremner
                   ` (2 subsequent siblings)
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

The hook directory configuration needs to be kept in synch with the
other configuration information, so add scaffolding to support this at
database opening time.
---
 lib/config.cc          | 3 +++
 lib/notmuch.h          | 1 +
 test/T590-libconfig.sh | 1 +
 3 files changed, 5 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 443dc3a6..4b115a07 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -390,6 +390,8 @@ _notmuch_config_key_to_string (notmuch_config_key_t key) {
     switch (key) {
     case NOTMUCH_CONFIG_DATABASE_PATH:
 	return "database.path";
+    case NOTMUCH_CONFIG_HOOK_DIR:
+	return "database.hook_dir";
     case NOTMUCH_CONFIG_EXCLUDE_TAGS:
 	return "search.exclude_tags";
     case NOTMUCH_CONFIG_NEW_TAGS:
@@ -428,6 +430,7 @@ _notmuch_config_default (void *ctx, notmuch_config_key_t key) {
 	return "inbox;unread";
     case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS:
 	return "true";
+    case NOTMUCH_CONFIG_HOOK_DIR:
     case NOTMUCH_CONFIG_NEW_IGNORE:
     case NOTMUCH_CONFIG_USER_NAME:
     case NOTMUCH_CONFIG_PRIMARY_EMAIL:
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 0f14b569..3c3fd487 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2461,6 +2461,7 @@ notmuch_config_list_destroy (notmuch_config_list_t *config_list);
 typedef enum _notmuch_config_key {
     NOTMUCH_CONFIG_FIRST,
     NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST,
+    NOTMUCH_CONFIG_HOOK_DIR,
     NOTMUCH_CONFIG_EXCLUDE_TAGS,
     NOTMUCH_CONFIG_NEW_TAGS,
     NOTMUCH_CONFIG_NEW_IGNORE,
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 97f8fdc7..57e0bbc4 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -333,6 +333,7 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 MAIL_DIR
+NULL
 
 inbox;unread
 NULL
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 36/38] lib/open: set HOOK_DIR on open
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (34 preceding siblings ...)
  2021-01-16 17:04 ` [PATCH 35/38] lib/config: add HOOK_DIR David Bremner
@ 2021-01-16 17:04 ` David Bremner
  2021-01-16 17:04 ` [PATCH 37/38] CLI: use configured hook directory David Bremner
  2021-01-16 17:04 ` [PATCH 38/38] doc: describe new config framework David Bremner
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is a simple two step path search.  Most error checking is
deferred until running the hooks.
---
 lib/open.cc            | 42 ++++++++++++++++++++++++++++++++++++++++++
 test/T590-libconfig.sh | 33 ++++++++++++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 1 deletion(-)

diff --git a/lib/open.cc b/lib/open.cc
index 92661271..eb7c8a01 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -67,6 +67,44 @@ _xdg_dir (void *ctx,
 			    profile_name);
 }
 
+static notmuch_status_t
+_choose_hook_dir (notmuch_database_t *notmuch,
+		  const char *profile,
+		  char **message)
+{
+    const char *config;
+    const char *hook_dir;
+    struct stat st;
+    int err;
+
+    hook_dir = notmuch_config_get (notmuch, NOTMUCH_CONFIG_HOOK_DIR);
+
+    if (hook_dir)
+	return NOTMUCH_STATUS_SUCCESS;
+
+    config = _xdg_dir (notmuch, "XDG_CONFIG_HOME", ".config", profile);
+    if (! config)
+	return  NOTMUCH_STATUS_PATH_ERROR;
+
+    hook_dir = talloc_asprintf (notmuch, "%s/hooks", config);
+
+    err = stat (hook_dir, &st);
+    if (err) {
+	if (errno == ENOENT) {
+	    const char *database_path = notmuch_database_get_path (notmuch);
+	    hook_dir = talloc_asprintf (notmuch, "%s/.notmuch/hooks", database_path);
+	} else {
+	    IGNORE_RESULT (asprintf (message, "Error: Cannot stat %s: %s.\n",
+				     hook_dir, strerror (errno)));
+	    return NOTMUCH_STATUS_FILE_ERROR;
+	}
+    }
+
+    _notmuch_config_cache (notmuch, NOTMUCH_CONFIG_HOOK_DIR, hook_dir);
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static notmuch_status_t
 _load_key_file (const char *path,
 		const char *profile,
@@ -300,6 +338,10 @@ notmuch_database_open_with_config (const char *database_path,
 	if (status)
 	    goto DONE;
 
+	status = _choose_hook_dir (notmuch, profile, &message);
+	if (status)
+	    goto DONE;
+
 	status = _notmuch_config_load_defaults (notmuch);
 	if (status)
 	    goto DONE;
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 57e0bbc4..c78ed204 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -201,6 +201,37 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+test_begin_subtest "NOTMUCH_CONFIG_HOOK_DIR: traditional"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+   const char *val = notmuch_config_get (db, NOTMUCH_CONFIG_HOOK_DIR);
+   printf("database.hook_dir = %s\n", val);
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+database.hook_dir = MAIL_DIR/.notmuch/hooks
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "NOTMUCH_CONFIG_HOOK_DIR: xdg"
+dir="${HOME}/.config/notmuch/default/hooks"
+mkdir -p $dir
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
+{
+   const char *val = notmuch_config_get (db, NOTMUCH_CONFIG_HOOK_DIR);
+   printf("database.hook_dir = %s\n", val);
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+database.hook_dir = CWD/home/.config/notmuch/default/hooks
+== stderr ==
+EOF
+rmdir $dir
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "notmuch_config_get_values"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
@@ -333,7 +364,7 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 MAIL_DIR
-NULL
+MAIL_DIR/.notmuch/hooks
 
 inbox;unread
 NULL
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 37/38] CLI: use configured hook directory
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (35 preceding siblings ...)
  2021-01-16 17:04 ` [PATCH 36/38] lib/open: set HOOK_DIR on open David Bremner
@ 2021-01-16 17:04 ` David Bremner
  2021-01-24 12:59   ` Tomi Ollila
  2021-01-16 17:04 ` [PATCH 38/38] doc: describe new config framework David Bremner
  37 siblings, 1 reply; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This enables support for hooks outside the database directory.
It relies strongly on configuration information being usable between
closing the database and destroying it.
---
 hooks.c            |   7 +-
 notmuch-client.h   |   2 +-
 notmuch-insert.c   |   7 +-
 notmuch-new.c      |   4 +-
 test/T400-hooks.sh | 197 +++++++++++++++++++++++++--------------------
 5 files changed, 120 insertions(+), 97 deletions(-)

diff --git a/hooks.c b/hooks.c
index 59c58070..ec89b22e 100644
--- a/hooks.c
+++ b/hooks.c
@@ -24,14 +24,15 @@
 #include <sys/wait.h>
 
 int
-notmuch_run_hook (const char *db_path, const char *hook)
+notmuch_run_hook (notmuch_database_t *notmuch, const char *hook)
 {
     char *hook_path;
     int status = 0;
     pid_t pid;
 
-    hook_path = talloc_asprintf (NULL, "%s/%s/%s/%s", db_path, ".notmuch",
-				 "hooks", hook);
+    hook_path = talloc_asprintf (notmuch, "%s/%s",
+				 notmuch_config_get (notmuch, NOTMUCH_CONFIG_HOOK_DIR),
+				 hook);
     if (hook_path == NULL) {
 	fprintf (stderr, "Out of memory\n");
 	return 1;
diff --git a/notmuch-client.h b/notmuch-client.h
index 9e09c36a..f60f5406 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -339,7 +339,7 @@ const char *
 _notmuch_config_get_path (notmuch_config_t *config);
 
 int
-notmuch_run_hook (const char *db_path, const char *hook);
+notmuch_run_hook (notmuch_database_t *notmuch, const char *hook);
 
 bool
 debugger_is_active (void);
diff --git a/notmuch-insert.c b/notmuch-insert.c
index e483b949..0f272e2e 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -481,7 +481,6 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
     notmuch_process_shared_options (argv[0]);
 
 
-    /* XXX TODO replace this use of DATABASE_PATH with something specific to hooks */
     db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH);
 
     if (! db_path)
@@ -570,7 +569,7 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
     status = add_file (notmuch, newpath, tag_ops, synchronize_flags, keep, indexing_cli_choices.opts);
 
     /* Commit changes. */
-    close_status = notmuch_database_destroy (notmuch);
+    close_status = notmuch_database_close (notmuch);
     if (close_status) {
 	/* Hold on to the first error, if any. */
 	if (! status)
@@ -595,9 +594,11 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
 
     if (hooks && status == NOTMUCH_STATUS_SUCCESS) {
 	/* Ignore hook failures. */
-	notmuch_run_hook (db_path, "post-insert");
+	notmuch_run_hook (notmuch, "post-insert");
     }
 
+    notmuch_database_destroy (notmuch);
+
     talloc_free (local);
 
     return status_to_exit (status);
diff --git a/notmuch-new.c b/notmuch-new.c
index 0f416939..2fc34e2c 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -1167,7 +1167,7 @@ notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmu
     }
 
     if (hooks) {
-	ret = notmuch_run_hook (db_path, "pre-new");
+	ret = notmuch_run_hook (notmuch, "pre-new");
 	if (ret)
 	    return EXIT_FAILURE;
     }
@@ -1284,7 +1284,7 @@ notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmu
     notmuch_database_close (notmuch);
 
     if (hooks && ! ret && ! interrupted)
-	ret = notmuch_run_hook (db_path, "post-new");
+	ret = notmuch_run_hook (notmuch, "post-new");
 
     notmuch_database_destroy (notmuch);
 
diff --git a/test/T400-hooks.sh b/test/T400-hooks.sh
index 49c690eb..b9894853 100755
--- a/test/T400-hooks.sh
+++ b/test/T400-hooks.sh
@@ -2,8 +2,6 @@
 test_description='hooks'
 . $(dirname "$0")/test-lib.sh || exit 1
 
-HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
-
 create_echo_hook () {
     local TOKEN="${RANDOM}"
     mkdir -p ${HOOK_DIR}
@@ -16,6 +14,7 @@ EOF
 }
 
 create_failing_hook () {
+    local HOOK_DIR=${2}
     mkdir -p ${HOOK_DIR}
     cat <<EOF >"${HOOK_DIR}/${1}"
 #!/bin/sh
@@ -24,98 +23,120 @@ EOF
     chmod +x "${HOOK_DIR}/${1}"
 }
 
-rm_hooks () {
-    rm -rf ${HOOK_DIR}
-}
-
 # add a message to generate mail dir and database
 add_message
 # create maildir structure for notmuch-insert
 mkdir -p "$MAIL_DIR"/{cur,new,tmp}
 
-test_begin_subtest "pre-new is run"
-rm_hooks
-generate_message
-create_echo_hook "pre-new" expected output
-notmuch new > /dev/null
-test_expect_equal_file expected output
-
-test_begin_subtest "post-new is run"
-rm_hooks
-generate_message
-create_echo_hook "post-new" expected output
-notmuch new > /dev/null
-test_expect_equal_file expected output
-
-test_begin_subtest "post-insert hook is run"
-rm_hooks
-generate_message
-create_echo_hook "post-insert" expected output
-notmuch insert < "$gen_msg_filename"
-test_expect_equal_file expected output
-
-test_begin_subtest "pre-new is run before post-new"
-rm_hooks
-generate_message
-create_echo_hook "pre-new" pre-new.expected pre-new.output
-create_echo_hook "post-new" post-new.expected post-new.output
-notmuch new > /dev/null
-test_expect_equal_file post-new.expected post-new.output
-
-test_begin_subtest "pre-new non-zero exit status (hook status)"
-rm_hooks
-generate_message
-create_failing_hook "pre-new"
-output=`notmuch new 2>&1`
-test_expect_equal "$output" "Error: pre-new hook failed with status 13"
-
-# depends on the previous subtest leaving broken hook behind
-test_begin_subtest "pre-new non-zero exit status (notmuch status)"
-test_expect_code 1 "notmuch new"
-
-# depends on the previous subtests leaving 1 new message behind
-test_begin_subtest "pre-new non-zero exit status aborts new"
-rm_hooks
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "Added 1 new message to the database."
-
-test_begin_subtest "post-new non-zero exit status (hook status)"
-rm_hooks
-generate_message
-create_failing_hook "post-new"
-NOTMUCH_NEW 2>output.stderr >output
-cat output.stderr >> output
-echo "Added 1 new message to the database." > expected
-echo "Error: post-new hook failed with status 13" >> expected
-test_expect_equal_file expected output
-
-# depends on the previous subtest leaving broken hook behind
-test_begin_subtest "post-new non-zero exit status (notmuch status)"
-test_expect_code 1 "notmuch new"
-
-test_begin_subtest "post-insert hook does not affect insert status"
-rm_hooks
-generate_message
-create_failing_hook "post-insert"
-test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null"
-
-test_begin_subtest "hook without executable permissions"
-rm_hooks
-mkdir -p ${HOOK_DIR}
-cat <<EOF >"${HOOK_DIR}/pre-new"
-#!/bin/sh
-echo foo
+for config in traditional profile explicit XDG; do
+    unset NOTMUCH_PROFILE
+    notmuch config set database.hook_dir
+    case $config in
+	traditional)
+	    HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
+	    ;;
+	profile)
+	    dir=${HOME}/.config/notmuch/other
+	    mkdir -p ${dir}
+	    HOOK_DIR=${dir}/hooks
+	    cp ${NOTMUCH_CONFIG} ${dir}/config
+	    export NOTMUCH_PROFILE=other
+	    ;;
+	explicit)
+	    HOOK_DIR=${HOME}/.notmuch-hooks
+	    mkdir -p $HOOK_DIR
+	    notmuch config set database.hook_dir $HOOK_DIR
+	    ;;
+	*)
+	    HOOK_DIR=${HOME}/.config/notmuch/default/hooks
+	    ;;
+    esac
+
+    test_begin_subtest "pre-new is run [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "pre-new" expected output $HOOK_DIR
+    notmuch new > /dev/null
+    test_expect_equal_file expected output
+
+    test_begin_subtest "post-new is run [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "post-new" expected output $HOOK_DIR
+    notmuch new > /dev/null
+    test_expect_equal_file expected output
+
+    test_begin_subtest "post-insert hook is run [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "post-insert" expected output $HOOK_DIR
+    notmuch insert < "$gen_msg_filename"
+    test_expect_equal_file expected output
+
+    test_begin_subtest "pre-new is run before post-new [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_echo_hook "pre-new" pre-new.expected pre-new.output $HOOK_DIR
+    create_echo_hook "post-new" post-new.expected post-new.output $HOOK_DIR
+    notmuch new > /dev/null
+    test_expect_equal_file post-new.expected post-new.output
+
+    test_begin_subtest "pre-new non-zero exit status (hook status) [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_failing_hook "pre-new" $HOOK_DIR
+    output=`notmuch new 2>&1`
+    test_expect_equal "$output" "Error: pre-new hook failed with status 13"
+
+    # depends on the previous subtest leaving broken hook behind
+    test_begin_subtest "pre-new non-zero exit status (notmuch status) [${config}]"
+    test_expect_code 1 "notmuch new"
+
+    # depends on the previous subtests leaving 1 new message behind
+    test_begin_subtest "pre-new non-zero exit status aborts new [${config}]"
+    rm -rf ${HOOK_DIR}
+    output=$(NOTMUCH_NEW)
+    test_expect_equal "$output" "Added 1 new message to the database."
+
+    test_begin_subtest "post-new non-zero exit status (hook status) [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_failing_hook "post-new" $HOOK_DIR
+    NOTMUCH_NEW 2>output.stderr >output
+    cat output.stderr >> output
+    echo "Added 1 new message to the database." > expected
+    echo "Error: post-new hook failed with status 13" >> expected
+    test_expect_equal_file expected output
+
+    # depends on the previous subtest leaving broken hook behind
+    test_begin_subtest "post-new non-zero exit status (notmuch status) [${config}]"
+    test_expect_code 1 "notmuch new"
+
+    test_begin_subtest "post-insert hook does not affect insert status [${config}]"
+    rm -rf ${HOOK_DIR}
+    generate_message
+    create_failing_hook "post-insert" $HOOK_DIR
+    test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null"
+
+    test_begin_subtest "hook without executable permissions [${config}]"
+    rm -rf ${HOOK_DIR}
+    mkdir -p ${HOOK_DIR}
+    cat <<EOF >"${HOOK_DIR}/pre-new"
+    #!/bin/sh
+    echo foo
 EOF
-output=`notmuch new 2>&1`
-test_expect_code 1 "notmuch new"
-
-test_begin_subtest "hook execution failure"
-rm_hooks
-mkdir -p ${HOOK_DIR}
-cat <<EOF >"${HOOK_DIR}/pre-new"
-no hashbang, execl fails
+    output=`notmuch new 2>&1`
+    test_expect_code 1 "notmuch new"
+
+    test_begin_subtest "hook execution failure [${config}]"
+    rm -rf ${HOOK_DIR}
+    mkdir -p ${HOOK_DIR}
+    cat <<EOF >"${HOOK_DIR}/pre-new"
+    no hashbang, execl fails
 EOF
-chmod +x "${HOOK_DIR}/pre-new"
-test_expect_code 1 "notmuch new"
+    chmod +x "${HOOK_DIR}/pre-new"
+    test_expect_code 1 "notmuch new"
 
+    rm -rf ${HOOK_DIR}
+done
 test_done
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 38/38] doc: describe new config framework
  2021-01-16 17:03 v4 merged config David Bremner
                   ` (36 preceding siblings ...)
  2021-01-16 17:04 ` [PATCH 37/38] CLI: use configured hook directory David Bremner
@ 2021-01-16 17:04 ` David Bremner
  37 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-16 17:04 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Remove STORED IN DATABASE discussion, describe search rules.
Currently profiles seem a bit pointless. They will make more sense
when they apply to databases as well.
---
 doc/man1/notmuch-config.rst | 43 +++++++++++++++++++++++++++++--------
 doc/man5/notmuch-hooks.rst  |  6 +++---
 2 files changed, 37 insertions(+), 12 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 769f336a..bc597957 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -17,10 +17,6 @@ DESCRIPTION
 The **config** command can be used to get or set settings in the notmuch
 configuration file and corresponding database.
 
-Items marked **[STORED IN DATABASE]** are only in the database.  They
-should not be placed in the configuration file, and should be accessed
-programmatically as described in the SYNOPSIS above.
-
 **get**
     The value of the specified configuration item is printed to
     stdout. If the item has multiple values (it is a list), each value
@@ -54,6 +50,11 @@ The available configuration items are described below.
 
     Default: ``$MAILDIR`` variable if set, otherwise ``$HOME/mail``.
 
+**database.hook_dir**
+
+    Directory containing hooks run by notmuch commands. See
+    **notmuch-hooks(5)**.
+
 **user.name**
     Your full name.
 
@@ -134,7 +135,7 @@ The available configuration items are described below.
 
     Default: ``true``.
 
-**index.decrypt** **[STORED IN DATABASE]**
+**index.decrypt**
     Policy for decrypting encrypted messages during indexing.  Must be
     one of: ``false``, ``auto``, ``nostash``, or ``true``.
 
@@ -187,7 +188,7 @@ The available configuration items are described below.
 
     Default: ``auto``.
 
-**index.header.<prefix>** **[STORED IN DATABASE]**
+**index.header.<prefix>**
     Define the query prefix <prefix>, based on a mail header. For
     example ``index.header.List=List-Id`` will add a probabilistic
     prefix ``List:`` that searches the ``List-Id`` field.  User
@@ -202,7 +203,7 @@ The available configuration items are described below.
     (since notmuch 0.30, "compact" and "field_processor" are
     always included.)
 
-**query.<name>** **[STORED IN DATABASE]**
+**query.<name>**
     Expansion for named query called <name>. See
     **notmuch-search-terms(7)** for more information about named
     queries.
@@ -214,8 +215,32 @@ The following environment variables can be used to control the behavior
 of notmuch.
 
 **NOTMUCH\_CONFIG**
-    Specifies the location of the notmuch configuration file. Notmuch
-    will use ${HOME}/.notmuch-config if this variable is not set.
+    Specifies the location of the notmuch configuration file.
+
+**NOTMUCH_PROFILE**
+    Selects among notmuch configurations.
+
+FILES
+=====
+
+CONFIGURATION
+-------------
+
+If ``NOTMUCH_CONFIG`` is unset, notmuch tries (in order)
+
+- ``$XDG_CONFIG_HOME/notmuch/<profile>/config`` where ``<profile>`` is
+  defined by ``$NOTMUCH_PROFILE`` or "default"
+- ``${HOME}/.notmuch-config<profile>`` where ``<profile>`` is
+  ``.$NOTMUCH_PROFILE`` or ""
+
+Hooks
+-----
+
+If ``database.hook_dir`` is unset, notmuch tries (in order)
+
+- ``$XDG_CONFIG_HOME/notmuch/<profile>/hooks`` where ``<profile>`` is
+  defined by ``$NOTMUCH_PROFILE`` or "default"
+- ``<database.path>/.notmuch/hooks``
 
 SEE ALSO
 ========
diff --git a/doc/man5/notmuch-hooks.rst b/doc/man5/notmuch-hooks.rst
index de2ed0c2..c509afb3 100644
--- a/doc/man5/notmuch-hooks.rst
+++ b/doc/man5/notmuch-hooks.rst
@@ -5,15 +5,15 @@ notmuch-hooks
 SYNOPSIS
 ========
 
-$DATABASEDIR/.notmuch/hooks/*
+<hook_dir>/{pre-new, post-new, post-insert}
 
 DESCRIPTION
 ===========
 
 Hooks are scripts (or arbitrary executables or symlinks to such) that
 notmuch invokes before and after certain actions. These scripts reside
-in the .notmuch/hooks directory within the database directory and must
-have executable permissions.
+in a directory defined as described in **notmuch-config(1)**. They
+must have executable permissions.
 
 The currently available hooks are described below.
 
-- 
2.29.2

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* Re: [PATCH 01/38] lib: add _notmuch_string_map_set
  2021-01-16 17:03 ` [PATCH 01/38] lib: add _notmuch_string_map_set David Bremner
@ 2021-01-24 11:28   ` Tomi Ollila
  2021-01-24 12:06     ` David Bremner
  0 siblings, 1 reply; 43+ messages in thread
From: Tomi Ollila @ 2021-01-24 11:28 UTC (permalink / raw)
  To: David Bremner, notmuch


On Sat, Jan 16 2021, David Bremner wrote:


First, is this patch series safe to be applied and put into use ? :D

I've looked this through once. So much stuff hard to figure out
if there is something broken. 

Most of the comments here and other patches are about style (line
break), but some (IIRC at least 2) which changes functionality a bit...

> This will be used (and tested) by the configuration caching code to be
> added in the next commit.
> ---
>  lib/notmuch-private.h |  5 +++++
>  lib/string-map.c      | 15 +++++++++++++++
>  2 files changed, 20 insertions(+)
>
> diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
> index 57ec7f72..51016b0b 100644
> --- a/lib/notmuch-private.h
> +++ b/lib/notmuch-private.h
> @@ -638,6 +638,11 @@ _notmuch_string_map_append (notmuch_string_map_t *map,
>  			    const char *key,
>  			    const char *value);
>  
> +void
> +_notmuch_string_map_set (notmuch_string_map_t *map,
> +			 const char *key,
> +			 const char *value);
> +
>  const char *
>  _notmuch_string_map_get (notmuch_string_map_t *map, const char *key);
>  
> diff --git a/lib/string-map.c b/lib/string-map.c
> index a88404c7..1f3215fb 100644
> --- a/lib/string-map.c
> +++ b/lib/string-map.c
> @@ -142,6 +142,21 @@ bsearch_first (notmuch_string_pair_t *array, size_t len, const char *key, bool e
>  	return NULL;
>  
>  }
> +void
> +_notmuch_string_map_set (notmuch_string_map_t *map, const char *key, const char *val)

The style in string-map.c (and matches the .h above) would indicate the abobe should be:

_notmuch_string_map_set (notmuch_string_map_t *map, 
                         const char *key, 
                         const char *val)

> +{
> +    notmuch_string_pair_t *pair;
> +
> +    /* this means that calling string_map_set invalidates iterators */
> +    _notmuch_string_map_sort (map);
> +    pair = bsearch_first (map->pairs, map->length, key, true);
> +    if (! pair)
> +       _notmuch_string_map_append (map, key, val);
> +    else {
> +       talloc_free (pair->value);
> +       pair->value = talloc_strdup (map->pairs, val);
> +    }
> +}
>  
>  const char *
>  _notmuch_string_map_get (notmuch_string_map_t *map, const char *key)
> -- 
> 2.29.2

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH 04/38] lib/open: add support for config profiles and default locations
  2021-01-16 17:03 ` [PATCH 04/38] lib/open: add support for config profiles and default locations David Bremner
@ 2021-01-24 11:58   ` Tomi Ollila
  0 siblings, 0 replies; 43+ messages in thread
From: Tomi Ollila @ 2021-01-24 11:58 UTC (permalink / raw)
  To: David Bremner, notmuch; +Cc: David Bremner

On Sat, Jan 16 2021, David Bremner wrote:

In id:20210116170406.842014-4-david@tethera.net there is missing space
after , 

    gchar **groups,**keys, *val;

Here, perhaps s/This commits fills in/Fill in/  (?)


> This commit fills in the remainder of the documented functionality for
> n_d_open_with_config with respect to config file location. Similar
> searching default locations of the database file still needs to be
> added.
> ---
>  lib/open.cc            |  95 +++++++++++++++++++++----
>  test/T590-libconfig.sh | 153 ++++++++++++++++++++++++++++++++++++++---
>  2 files changed, 227 insertions(+), 21 deletions(-)
>
> diff --git a/lib/open.cc b/lib/open.cc
> index 7acaea7b..76255283 100644
> --- a/lib/open.cc
> +++ b/lib/open.cc
> @@ -37,6 +37,82 @@ notmuch_database_open_verbose (const char *path,
>  					      database, status_string);
>  }
>  
> +static const char *
> +_xdg_dir (void *ctx,
> +	  const char *xdg_root_variable,
> +	  const char *xdg_prefix,
> +	  const char *profile_name)
> +{
> +    const char *xdg_root = getenv (xdg_root_variable);
> +    const char *home = getenv ("HOME");

Resolving HOME from env is not needed always, move it inside
next if() -- that will also make its scope smaller.

> +
> +    if (! xdg_root) {
> +	if (! home) return NULL;
> +
> +	xdg_root = talloc_asprintf (ctx,
> +				    "%s/%s",
> +				    home,
> +				    xdg_prefix);
> +    }
> +
> +    if (! profile_name)
> +	profile_name = getenv ("NOTMUCH_PROFILE");
> +
> +    if (! profile_name)
> +	profile_name = "default";
> +
> +    return talloc_asprintf (ctx,
> +			    "%s/notmuch/%s",
> +			    xdg_root,
> +			    profile_name);
> +}
> +
> +static notmuch_status_t
> +_load_key_file (const char *path,
> +		const char *profile,
> +		GKeyFile **key_file)
> +{
> +    notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
> +    void *local = talloc_new (NULL);
> +
> +    if (path && EMPTY_STRING (path))
> +	goto DONE;
> +
> +    if (! path)
> +	path = getenv ("NOTMUCH_CONFIG");
> +
> +    if (! path) {
> +	const char *dir = _xdg_dir (local, "XDG_CONFIG_HOME", ".config", profile);
> +
> +	if (dir) {
> +	    path = talloc_asprintf (local, "%s/config", dir);
> +	    if (access (path, R_OK) !=0)
> +		path = NULL;
> +	}
> +    }
> +
> +    if (! path) {
> +	const char *home = getenv ("HOME");
> +
> +	path = talloc_asprintf (local, "%s/.notmuch-config", home);
> +
> +	if (! profile)
> +	    profile = getenv ("NOTMUCH_PROFILE");
> +
> +	if (profile)
> +	    path = talloc_asprintf (local, "%s.%s", path, profile);
> +    }
> +
> +    *key_file = g_key_file_new ();
> +    if (! g_key_file_load_from_file (*key_file, path, G_KEY_FILE_NONE, NULL)) {
> +	status = NOTMUCH_STATUS_FILE_ERROR;
> +    }
> +
> +DONE:
> +    talloc_free (local);
> +    return status;
> +}
> +
>  notmuch_status_t
>  notmuch_database_open_with_config (const char *database_path,
>  				   notmuch_database_mode_t mode,
> @@ -49,7 +125,6 @@ notmuch_database_open_with_config (const char *database_path,
>      void *local = talloc_new (NULL);
>      notmuch_database_t *notmuch = NULL;
>      char *notmuch_path, *xapian_path, *incompat_features;
> -    char *configured_database_path = NULL;
>      char *message = NULL;
>      struct stat st;
>      int err;
> @@ -57,18 +132,14 @@ notmuch_database_open_with_config (const char *database_path,
>      GKeyFile *key_file = NULL;
>      static int initialized = 0;
>  
> -    /* XXX TODO: default locations for NULL case, handle profiles */
> -    if (config_path != NULL && ! EMPTY_STRING (config_path)) {
> -	key_file = g_key_file_new ();
> -	if (! g_key_file_load_from_file (key_file, config_path, G_KEY_FILE_NONE, NULL)) {
> -	    status = NOTMUCH_STATUS_FILE_ERROR;
> -	    goto DONE;
> -	}
> -	configured_database_path = g_key_file_get_value (key_file, "database", "path", NULL);
> +    status = _load_key_file (config_path, profile, &key_file);
> +    if (status) {
> +	message = strdup ("Error: cannot load config file");
> +	goto DONE;
>      }
> -
> -    if (database_path == NULL)
> -	database_path = configured_database_path;
> +	
> +    if (! database_path && key_file)
> +	database_path = g_key_file_get_value (key_file, "database", "path", NULL);
>  
>      if (database_path == NULL) {
>  	message = strdup ("Error: Cannot open a database for a NULL path.\n");
> diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
> index 5fb1bb87..2986284a 100755
> --- a/test/T590-libconfig.sh
> +++ b/test/T590-libconfig.sh
> @@ -15,14 +15,21 @@ int main (int argc, char** argv)
>     notmuch_database_t *db;
>     char *val;
>     notmuch_status_t stat;
> +   char *msg = NULL;
>  
> -   EXPECT0(notmuch_database_open_with_config (argv[1],
> +   for (int i=1; i<argc; i++)

spaces

> +      if (strcmp (argv[i], "%NULL%") == 0) argv[i] = NULL;
> +
> +   stat = notmuch_database_open_with_config (argv[1],
>                                                NOTMUCH_DATABASE_MODE_READ_WRITE,
>                                                argv[2],
> -                                              NULL,
> +                                              argv[3],
>                                                &db,
> -                                              NULL));
> -
> +                                              &msg);
> +   if (stat != NOTMUCH_STATUS_SUCCESS) {
> +     fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : "");
> +     exit (1);
> +   }
>  EOF
>  
>  cat <<EOF > c_tail
> @@ -51,7 +58,7 @@ test_expect_equal_file EXPECTED OUTPUT
>  
>  
>  test_begin_subtest "notmuch_database_get_config_list: empty list"
> -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
>  {
>     notmuch_config_list_t *list;
>     EXPECT0(notmuch_database_get_config_list (db, "nonexistent", &list));
> @@ -83,7 +90,7 @@ EOF
>  test_expect_equal_file EXPECTED OUTPUT
>  
>  test_begin_subtest "notmuch_database_get_config_list: all pairs"
> -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
>  {
>     notmuch_config_list_t *list;
>     EXPECT0(notmuch_database_set_config (db, "zzzafter", "afterval"));
> @@ -128,7 +135,7 @@ EOF
>  test_expect_equal_file EXPECTED OUTPUT
>  
>  test_begin_subtest "notmuch_database_get_config_list: one prefix"
> -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
>  {
>     notmuch_config_list_t *list;
>     EXPECT0(notmuch_database_get_config_list (db, "test.key", &list));
> @@ -147,7 +154,7 @@ EOF
>  test_expect_equal_file EXPECTED OUTPUT
>  
>  test_begin_subtest "dump config"
> -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
>  {
>      EXPECT0(notmuch_database_set_config (db, "key with spaces", "value, with, spaces!"));
>  }
> @@ -165,7 +172,7 @@ test_expect_equal_file EXPECTED OUTPUT
>  
>  test_begin_subtest "restore config"
>  notmuch dump --include=config >EXPECTED
> -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
>  {
>      EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
>  }
> @@ -194,4 +201,132 @@ EOF
>  test_expect_equal_file EXPECTED OUTPUT
>  restore_database
>  
> +backup_database
> +test_begin_subtest "override config from \${NOTMUCH_CONFIG}"
> +notmuch config set test.key1 overridden
> +# second argument omitted to make argv[2] == NULL
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
> +{
> +   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
> +   printf("test.key1 = %s\n", val);
> +   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
> +   printf("test.key2 = %s\n", val);
> +}
> +EOF
> +notmuch config set test.key1
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +test.key1 = overridden
> +test.key2 = testvalue2
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +restore_database
> +
> +backup_database
> +test_begin_subtest "override config from \${HOME}/.notmuch-config"
> +ovconfig=${HOME}/.notmuch-config
> +cp ${NOTMUCH_CONFIG} ${ovconfig}
> +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
> +unset NOTMUCH_CONFIG
> +notmuch --config=${ovconfig} config set test.key1 overridden-home
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL%
> +{
> +   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
> +   printf("test.key1 = %s\n", val);
> +   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
> +   printf("test.key2 = %s\n", val);
> +}
> +EOF
> +rm -f ${ovconfig}
> +NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +test.key1 = overridden-home
> +test.key2 = testvalue2
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +restore_database
> +
> +backup_database
> +test_begin_subtest "override config from \${XDG_CONFIG_HOME}/notmuch"
> +ovconfig=${HOME}/.config/notmuch/default/config
> +mkdir -p $(dirname ${ovconfig})
> +cp ${NOTMUCH_CONFIG} ${ovconfig}
> +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
> +unset NOTMUCH_CONFIG
> +notmuch --config=${ovconfig} config set test.key1 overridden-xdg
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL%
> +{
> +   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
> +   printf("test.key1 = %s\n", val);
> +   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
> +   printf("test.key2 = %s\n", val);
> +}
> +EOF
> +rm -f ${ovconfig}
> +NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +test.key1 = overridden-xdg
> +test.key2 = testvalue2
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +restore_database
> +
> +backup_database
> +test_begin_subtest "override config from \${XDG_CONFIG_HOME}/notmuch with profile"
> +ovconfig=${HOME}/.config/notmuch/work/config
> +mkdir -p $(dirname ${ovconfig})
> +cp ${NOTMUCH_CONFIG} ${ovconfig}
> +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
> +unset NOTMUCH_CONFIG
> +notmuch --config=${ovconfig} config set test.key1 overridden-xdg-profile
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% work
> +{
> +   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
> +   printf("test.key1 = %s\n", val);
> +   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
> +   printf("test.key2 = %s\n", val);
> +}
> +EOF
> +rm -f ${ovconfig}
> +NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +test.key1 = overridden-xdg-profile
> +test.key2 = testvalue2
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +restore_database
> +
> +backup_database
> +test_begin_subtest "override config from \${HOME}/.notmuch-config.work (via args)"

(left out comments where "s could be replaced w/ 's (and w/o \ before $))

> +ovconfig=${HOME}/.notmuch-config.work
> +cp ${NOTMUCH_CONFIG} ${ovconfig}
> +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG}
> +unset NOTMUCH_CONFIG
> +notmuch --config=${ovconfig} config set test.key1 overridden-profile
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% work
> +{
> +   EXPECT0(notmuch_database_get_config (db, "test.key1", &val));
> +   printf("test.key1 = %s\n", val);
> +   EXPECT0(notmuch_database_get_config (db, "test.key2", &val));
> +   printf("test.key2 = %s\n", val);
> +}
> +EOF
> +#rm -f ${ovconfig}
> +NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG}
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +test.key1 = overridden-profile
> +test.key2 = testvalue2
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +restore_database
> +
>  test_done
> -- 
> 2.29.2

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH 01/38] lib: add _notmuch_string_map_set
  2021-01-24 11:28   ` Tomi Ollila
@ 2021-01-24 12:06     ` David Bremner
  0 siblings, 0 replies; 43+ messages in thread
From: David Bremner @ 2021-01-24 12:06 UTC (permalink / raw)
  To: Tomi Ollila, notmuch

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Sat, Jan 16 2021, David Bremner wrote:
>
>
> First, is this patch series safe to be applied and put into use ? :D

At least the backwards compatibility part is pretty well tested. I have
been running variations of it for a month or so.

>
> The style in string-map.c (and matches the .h above) would indicate the abobe should be:
>
> _notmuch_string_map_set (notmuch_string_map_t *map, 
>                          const char *key, 
>                          const char *val)
>

Sure. Although I think string-map.c is not completely consistent, that
is no reason not to match the header for new stuff.

d

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH 37/38] CLI: use configured hook directory
  2021-01-16 17:04 ` [PATCH 37/38] CLI: use configured hook directory David Bremner
@ 2021-01-24 12:59   ` Tomi Ollila
  0 siblings, 0 replies; 43+ messages in thread
From: Tomi Ollila @ 2021-01-24 12:59 UTC (permalink / raw)
  To: David Bremner, notmuch; +Cc: David Bremner

On Sat, Jan 16 2021, David Bremner wrote:

> This enables support for hooks outside the database directory.
> It relies strongly on configuration information being usable between
> closing the database and destroying it.
> ---
>  hooks.c            |   7 +-
>  notmuch-client.h   |   2 +-
>  notmuch-insert.c   |   7 +-
>  notmuch-new.c      |   4 +-
>  test/T400-hooks.sh | 197 +++++++++++++++++++++++++--------------------
>  5 files changed, 120 insertions(+), 97 deletions(-)
>
> diff --git a/hooks.c b/hooks.c
> index 59c58070..ec89b22e 100644
> --- a/hooks.c
> +++ b/hooks.c
> @@ -24,14 +24,15 @@
>  #include <sys/wait.h>
>  
>  int
> -notmuch_run_hook (const char *db_path, const char *hook)
> +notmuch_run_hook (notmuch_database_t *notmuch, const char *hook)
>  {
>      char *hook_path;
>      int status = 0;
>      pid_t pid;
>  
> -    hook_path = talloc_asprintf (NULL, "%s/%s/%s/%s", db_path, ".notmuch",
> -				 "hooks", hook);
> +    hook_path = talloc_asprintf (notmuch, "%s/%s",
> +				 notmuch_config_get (notmuch, NOTMUCH_CONFIG_HOOK_DIR),
> +				 hook);
>      if (hook_path == NULL) {
>  	fprintf (stderr, "Out of memory\n");
>  	return 1;
> diff --git a/notmuch-client.h b/notmuch-client.h
> index 9e09c36a..f60f5406 100644
> --- a/notmuch-client.h
> +++ b/notmuch-client.h
> @@ -339,7 +339,7 @@ const char *
>  _notmuch_config_get_path (notmuch_config_t *config);
>  
>  int
> -notmuch_run_hook (const char *db_path, const char *hook);
> +notmuch_run_hook (notmuch_database_t *notmuch, const char *hook);
>  
>  bool
>  debugger_is_active (void);
> diff --git a/notmuch-insert.c b/notmuch-insert.c
> index e483b949..0f272e2e 100644
> --- a/notmuch-insert.c
> +++ b/notmuch-insert.c
> @@ -481,7 +481,6 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
>      notmuch_process_shared_options (argv[0]);
>  
>  
> -    /* XXX TODO replace this use of DATABASE_PATH with something specific to hooks */
>      db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH);
>  
>      if (! db_path)
> @@ -570,7 +569,7 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
>      status = add_file (notmuch, newpath, tag_ops, synchronize_flags, keep, indexing_cli_choices.opts);
>  
>      /* Commit changes. */
> -    close_status = notmuch_database_destroy (notmuch);
> +    close_status = notmuch_database_close (notmuch);
>      if (close_status) {
>  	/* Hold on to the first error, if any. */
>  	if (! status)
> @@ -595,9 +594,11 @@ notmuch_insert_command (unused(notmuch_config_t *config),notmuch_database_t *not
>  
>      if (hooks && status == NOTMUCH_STATUS_SUCCESS) {
>  	/* Ignore hook failures. */
> -	notmuch_run_hook (db_path, "post-insert");
> +	notmuch_run_hook (notmuch, "post-insert");
>      }
>  
> +    notmuch_database_destroy (notmuch);
> +
>      talloc_free (local);
>  
>      return status_to_exit (status);
> diff --git a/notmuch-new.c b/notmuch-new.c
> index 0f416939..2fc34e2c 100644
> --- a/notmuch-new.c
> +++ b/notmuch-new.c
> @@ -1167,7 +1167,7 @@ notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmu
>      }
>  
>      if (hooks) {
> -	ret = notmuch_run_hook (db_path, "pre-new");
> +	ret = notmuch_run_hook (notmuch, "pre-new");
>  	if (ret)
>  	    return EXIT_FAILURE;
>      }
> @@ -1284,7 +1284,7 @@ notmuch_new_command (unused(notmuch_config_t *config), notmuch_database_t *notmu
>      notmuch_database_close (notmuch);
>  
>      if (hooks && ! ret && ! interrupted)
> -	ret = notmuch_run_hook (db_path, "post-new");
> +	ret = notmuch_run_hook (notmuch, "post-new");
>  
>      notmuch_database_destroy (notmuch);
>  
> diff --git a/test/T400-hooks.sh b/test/T400-hooks.sh
> index 49c690eb..b9894853 100755
> --- a/test/T400-hooks.sh
> +++ b/test/T400-hooks.sh
> @@ -2,8 +2,6 @@
>  test_description='hooks'
>  . $(dirname "$0")/test-lib.sh || exit 1
>  
> -HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
> -
>  create_echo_hook () {
>      local TOKEN="${RANDOM}"
>      mkdir -p ${HOOK_DIR}
> @@ -16,6 +14,7 @@ EOF
>  }
>  
>  create_failing_hook () {
> +    local HOOK_DIR=${2}
>      mkdir -p ${HOOK_DIR}
>      cat <<EOF >"${HOOK_DIR}/${1}"
>  #!/bin/sh
> @@ -24,98 +23,120 @@ EOF
>      chmod +x "${HOOK_DIR}/${1}"
>  }
>  
> -rm_hooks () {
> -    rm -rf ${HOOK_DIR}
> -}
> -
>  # add a message to generate mail dir and database
>  add_message
>  # create maildir structure for notmuch-insert
>  mkdir -p "$MAIL_DIR"/{cur,new,tmp}
>  
> -test_begin_subtest "pre-new is run"
> -rm_hooks
> -generate_message
> -create_echo_hook "pre-new" expected output
> -notmuch new > /dev/null
> -test_expect_equal_file expected output
> -
> -test_begin_subtest "post-new is run"
> -rm_hooks
> -generate_message
> -create_echo_hook "post-new" expected output
> -notmuch new > /dev/null
> -test_expect_equal_file expected output
> -
> -test_begin_subtest "post-insert hook is run"
> -rm_hooks
> -generate_message
> -create_echo_hook "post-insert" expected output
> -notmuch insert < "$gen_msg_filename"
> -test_expect_equal_file expected output
> -
> -test_begin_subtest "pre-new is run before post-new"
> -rm_hooks
> -generate_message
> -create_echo_hook "pre-new" pre-new.expected pre-new.output
> -create_echo_hook "post-new" post-new.expected post-new.output
> -notmuch new > /dev/null
> -test_expect_equal_file post-new.expected post-new.output
> -
> -test_begin_subtest "pre-new non-zero exit status (hook status)"
> -rm_hooks
> -generate_message
> -create_failing_hook "pre-new"
> -output=`notmuch new 2>&1`
> -test_expect_equal "$output" "Error: pre-new hook failed with status 13"
> -
> -# depends on the previous subtest leaving broken hook behind
> -test_begin_subtest "pre-new non-zero exit status (notmuch status)"
> -test_expect_code 1 "notmuch new"
> -
> -# depends on the previous subtests leaving 1 new message behind
> -test_begin_subtest "pre-new non-zero exit status aborts new"
> -rm_hooks
> -output=$(NOTMUCH_NEW)
> -test_expect_equal "$output" "Added 1 new message to the database."
> -
> -test_begin_subtest "post-new non-zero exit status (hook status)"
> -rm_hooks
> -generate_message
> -create_failing_hook "post-new"
> -NOTMUCH_NEW 2>output.stderr >output
> -cat output.stderr >> output
> -echo "Added 1 new message to the database." > expected
> -echo "Error: post-new hook failed with status 13" >> expected
> -test_expect_equal_file expected output
> -
> -# depends on the previous subtest leaving broken hook behind
> -test_begin_subtest "post-new non-zero exit status (notmuch status)"
> -test_expect_code 1 "notmuch new"
> -
> -test_begin_subtest "post-insert hook does not affect insert status"
> -rm_hooks
> -generate_message
> -create_failing_hook "post-insert"
> -test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null"
> -
> -test_begin_subtest "hook without executable permissions"
> -rm_hooks
> -mkdir -p ${HOOK_DIR}
> -cat <<EOF >"${HOOK_DIR}/pre-new"
> -#!/bin/sh
> -echo foo
> +for config in traditional profile explicit XDG; do
> +    unset NOTMUCH_PROFILE
> +    notmuch config set database.hook_dir
> +    case $config in
> +	traditional)
> +	    HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
> +	    ;;
> +	profile)
> +	    dir=${HOME}/.config/notmuch/other
> +	    mkdir -p ${dir}
> +	    HOOK_DIR=${dir}/hooks
> +	    cp ${NOTMUCH_CONFIG} ${dir}/config
> +	    export NOTMUCH_PROFILE=other
> +	    ;;
> +	explicit)
> +	    HOOK_DIR=${HOME}/.notmuch-hooks
> +	    mkdir -p $HOOK_DIR
> +	    notmuch config set database.hook_dir $HOOK_DIR
> +	    ;;
> +	*)

This '*' would better be 'XDG', to match the loop value.

As this is not user input, '*' (with error message)
can be left out.

This concludes this review; I left out some comments about
long lines as those were consisten w/ each other, and
there were inconsistensiens in source files there. Also
there were one '{' at the end of a function definition,
but that file had one other file like that so...

Like I said before I did not see anything that could break
things (I suppose current configs work (for now)), but there
are so many changes it is hard to figure out everything...

Tomi

> +	    HOOK_DIR=${HOME}/.config/notmuch/default/hooks
> +	    ;;
> +    esac
> +
> +    test_begin_subtest "pre-new is run [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_echo_hook "pre-new" expected output $HOOK_DIR
> +    notmuch new > /dev/null
> +    test_expect_equal_file expected output
> +
> +    test_begin_subtest "post-new is run [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_echo_hook "post-new" expected output $HOOK_DIR
> +    notmuch new > /dev/null
> +    test_expect_equal_file expected output
> +
> +    test_begin_subtest "post-insert hook is run [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_echo_hook "post-insert" expected output $HOOK_DIR
> +    notmuch insert < "$gen_msg_filename"
> +    test_expect_equal_file expected output
> +
> +    test_begin_subtest "pre-new is run before post-new [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_echo_hook "pre-new" pre-new.expected pre-new.output $HOOK_DIR
> +    create_echo_hook "post-new" post-new.expected post-new.output $HOOK_DIR
> +    notmuch new > /dev/null
> +    test_expect_equal_file post-new.expected post-new.output
> +
> +    test_begin_subtest "pre-new non-zero exit status (hook status) [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_failing_hook "pre-new" $HOOK_DIR
> +    output=`notmuch new 2>&1`
> +    test_expect_equal "$output" "Error: pre-new hook failed with status 13"
> +
> +    # depends on the previous subtest leaving broken hook behind
> +    test_begin_subtest "pre-new non-zero exit status (notmuch status) [${config}]"
> +    test_expect_code 1 "notmuch new"
> +
> +    # depends on the previous subtests leaving 1 new message behind
> +    test_begin_subtest "pre-new non-zero exit status aborts new [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    output=$(NOTMUCH_NEW)
> +    test_expect_equal "$output" "Added 1 new message to the database."
> +
> +    test_begin_subtest "post-new non-zero exit status (hook status) [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_failing_hook "post-new" $HOOK_DIR
> +    NOTMUCH_NEW 2>output.stderr >output
> +    cat output.stderr >> output
> +    echo "Added 1 new message to the database." > expected
> +    echo "Error: post-new hook failed with status 13" >> expected
> +    test_expect_equal_file expected output
> +
> +    # depends on the previous subtest leaving broken hook behind
> +    test_begin_subtest "post-new non-zero exit status (notmuch status) [${config}]"
> +    test_expect_code 1 "notmuch new"
> +
> +    test_begin_subtest "post-insert hook does not affect insert status [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    generate_message
> +    create_failing_hook "post-insert" $HOOK_DIR
> +    test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null"
> +
> +    test_begin_subtest "hook without executable permissions [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    mkdir -p ${HOOK_DIR}
> +    cat <<EOF >"${HOOK_DIR}/pre-new"
> +    #!/bin/sh
> +    echo foo
>  EOF
> -output=`notmuch new 2>&1`
> -test_expect_code 1 "notmuch new"
> -
> -test_begin_subtest "hook execution failure"
> -rm_hooks
> -mkdir -p ${HOOK_DIR}
> -cat <<EOF >"${HOOK_DIR}/pre-new"
> -no hashbang, execl fails
> +    output=`notmuch new 2>&1`
> +    test_expect_code 1 "notmuch new"
> +
> +    test_begin_subtest "hook execution failure [${config}]"
> +    rm -rf ${HOOK_DIR}
> +    mkdir -p ${HOOK_DIR}
> +    cat <<EOF >"${HOOK_DIR}/pre-new"
> +    no hashbang, execl fails
>  EOF
> -chmod +x "${HOOK_DIR}/pre-new"
> -test_expect_code 1 "notmuch new"
> +    chmod +x "${HOOK_DIR}/pre-new"
> +    test_expect_code 1 "notmuch new"
>  
> +    rm -rf ${HOOK_DIR}
> +done
>  test_done
> -- 
> 2.29.2

^ permalink raw reply	[flat|nested] 43+ messages in thread

end of thread, other threads:[~2021-01-24 12:59 UTC | newest]

Thread overview: 43+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-16 17:03 v4 merged config David Bremner
2021-01-16 17:03 ` [PATCH 01/38] lib: add _notmuch_string_map_set David Bremner
2021-01-24 11:28   ` Tomi Ollila
2021-01-24 12:06     ` David Bremner
2021-01-16 17:03 ` [PATCH 02/38] lib: cache configuration information from database David Bremner
2021-01-16 17:03 ` [PATCH 03/38] lib: add stub for notmuch_database_open_with_config David Bremner
2021-01-16 17:03 ` [PATCH 04/38] lib/open: add support for config profiles and default locations David Bremner
2021-01-24 11:58   ` Tomi Ollila
2021-01-16 17:03 ` [PATCH 05/38] CLI: generalize notmuch_config_mode_t David Bremner
2021-01-16 17:03 ` [PATCH 06/38] lib/config: add notmuch_config_key_{get,set} David Bremner
2021-01-16 17:03 ` [PATCH 07/38] lib/open: load default values for known configuration keys David Bremner
2021-01-16 17:03 ` [PATCH 08/38] CLI: add (unused) database argument to subcommands David Bremner
2021-01-16 17:03 ` [PATCH 09/38] util: add strsplit_len: simplified strtok with delimiter escaping David Bremner
2021-01-16 17:03 ` [PATCH 10/38] lib/config: add config values iterator David Bremner
2021-01-16 17:03 ` [PATCH 11/38] lib/config: make values iterators restartable David Bremner
2021-01-16 17:03 ` [PATCH 12/38] CLI/count: switch to new configuration framework David Bremner
2021-01-16 17:03 ` [PATCH 13/38] cli/dump: convert to new config framework David Bremner
2021-01-16 17:03 ` [PATCH 14/38] lib: add notmuch_config_get_bool David Bremner
2021-01-16 17:03 ` [PATCH 15/38] CLI/restore: convert to new config framework David Bremner
2021-01-16 17:03 ` [PATCH 16/38] CLI/insert: " David Bremner
2021-01-16 17:03 ` [PATCH 17/38] cli/reindex: convert " David Bremner
2021-01-16 17:03 ` [PATCH 18/38] CLI/reply: convert to " David Bremner
2021-01-16 17:03 ` [PATCH 19/38] CLI/{search,address}: convert to new configuration framework David Bremner
2021-01-16 17:03 ` [PATCH 20/38] cli/config: add accessor for config file name David Bremner
2021-01-16 17:03 ` [PATCH 21/38] CLI/show: mostly switch show to new config framework David Bremner
2021-01-16 17:03 ` [PATCH 22/38] cli/tag: convert " David Bremner
2021-01-16 17:03 ` [PATCH 23/38] lib/config: add _notmuch_config_cache David Bremner
2021-01-16 17:03 ` [PATCH 24/38] lib: split notmuch_database_compact David Bremner
2021-01-16 17:03 ` [PATCH 25/38] cli/compact: convert to new configuration framework David Bremner
2021-01-16 17:03 ` [PATCH 26/38] bindings/notmuch2: add missing crypto error status codes David Bremner
2021-01-16 17:03 ` [PATCH 27/38] lib/config: add NOTMUCH_CONFIG_NEW_IGNORE David Bremner
2021-01-16 17:03 ` [PATCH 28/38] lib/open: factor out first part of open David Bremner
2021-01-16 17:03 ` [PATCH 29/38] lib: add NOTMUCH_STATUS_NO_CONFIG David Bremner
2021-01-16 17:03 ` [PATCH 30/38] lib/database: move n_d_create* to open.cc David Bremner
2021-01-16 17:03 ` [PATCH 31/38] lib: add NOTMUCH_STATUS_DATABASE_EXISTS David Bremner
2021-01-16 17:04 ` [PATCH 32/38] lib: introduce notmuch_database_create_with_config David Bremner
2021-01-16 17:04 ` [PATCH 33/38] cli/new: refactor database upgrade code David Bremner
2021-01-16 17:04 ` [PATCH 34/38] cli/new: convert to new config framework David Bremner
2021-01-16 17:04 ` [PATCH 35/38] lib/config: add HOOK_DIR David Bremner
2021-01-16 17:04 ` [PATCH 36/38] lib/open: set HOOK_DIR on open David Bremner
2021-01-16 17:04 ` [PATCH 37/38] CLI: use configured hook directory David Bremner
2021-01-24 12:59   ` Tomi Ollila
2021-01-16 17:04 ` [PATCH 38/38] doc: describe new config framework David Bremner

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).