unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [kisara.moe] notmuch doesn't respect MAILDIR env for database path
@ 2021-05-08 20:49 mohsin kaleem
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
  0 siblings, 1 reply; 11+ messages in thread
From: mohsin kaleem @ 2021-05-08 20:49 UTC (permalink / raw)
  To: notmuch


notmuch doesn't seem to use the value of the $MAILDIR option when
assigning `database.path`.

On a new install from notmuch version 0.32 I ran `notmuch` to initialise
a config and then `notmuch new` which produces the output

```
Found 4 total files (that's not much mail).
Note: Ignoring non-mail file: /home/mohkale/.local/share/notmuch/default/xapian/flintlock
Note: Ignoring non-mail file: /home/mohkale/.local/share/notmuch/default/xapian/termlist.glass
Note: Ignoring non-mail file: /home/mohkale/.local/share/notmuch/default/xapian/postlist.glass
Note: Ignoring non-mail file: /home/mohkale/.local/share/notmuch/default/xapian/iamglass
Processed 4 total files in almost no time.
No new mail.
```

notmuch seems to default to using ~/.local/share/notmuch for the
`database.path` value instead of MAILDIR like the manual for
notmuch-config states.

S.N. My value of $MAILDIR is `~/docs/mail`. It exists in the maildir
format and has several mail files available already.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* look in MAILDIR for database
  2021-05-08 20:49 [kisara.moe] notmuch doesn't respect MAILDIR env for database path mohsin kaleem
@ 2021-08-02 15:09 ` David Bremner
  2021-08-02 15:09   ` [PATCH 1/8] test/path-config: define config_subtest David Bremner
                     ` (7 more replies)
  0 siblings, 8 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch

0.32 introduced a regression where notmuch stopped finding the database of people who specified it via Maildir.

The actual code fix is in

     [PATCH 7/8] lib/open: look in MAILDIR for database, as documented.

Also user facing is the updates to documentation in

     [PATCH 8/8] doc: document database search algorithm.

The rest of the series is tweaks to the test framework and some regression tests.

I'm thinking it's probably time to do another feature release, rather
than a point release just for this series.

d

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/8] test/path-config: define config_subtest
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-03 20:26     ` Tomi Ollila
  2021-08-02 15:09   ` [PATCH 2/8] test/expect_equal_file: whitespace cleanup David Bremner
                     ` (6 subsequent siblings)
  7 siblings, 1 reply; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This will allow adding "hook code" to be run for a particular
configuration, in particular marking all tests in a given
configuration as broken.
---
 test/T055-path-config.sh | 48 ++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 19 deletions(-)

diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh
index bb3bf665..2061b116 100755
--- a/test/T055-path-config.sh
+++ b/test/T055-path-config.sh
@@ -79,6 +79,13 @@ xdg_config () {
     notmuch --config=${CONFIG_PATH} config set database.path
 }
 
+config_subtest () {
+    local title=$1
+    local config=$2
+
+    test_begin_subtest "$title ($config)"
+}
+
 for config in traditional split XDG XDG+profile symlink home_mail; do
     #start each set of tests with an known set of messages
     add_email_corpus
@@ -106,20 +113,23 @@ for config in traditional split XDG XDG+profile symlink home_mail; do
 	home_mail)
 	    home_mail_config
 	    ;;
+	maildir_env)
+	    maildir_env_config
+	    ;;
     esac
 
-    test_begin_subtest "count ($config)"
+    config_subtest "count" "$config"
     output=$(notmuch count '*')
     test_expect_equal "$output" '52'
 
-    test_begin_subtest "count+tag ($config)"
+    config_subtest "count+tag" "$config"
     tag="tag${RANDOM}"
     notmuch tag +$tag '*'
     output=$(notmuch count tag:$tag)
     notmuch tag -$tag '*'
     test_expect_equal "$output" '52'
 
-    test_begin_subtest "address ($config)"
+    config_subtest "address" "$config"
     notmuch address --deduplicate=no --sort=newest-first --output=sender --output=recipients path:foo >OUTPUT
     cat <<EOF >EXPECTED
 Carl Worth <cworth@cworth.org>
@@ -127,7 +137,7 @@ notmuch@notmuchmail.org
 EOF
     test_expect_equal_file EXPECTED OUTPUT
 
-    test_begin_subtest "dump ($config)"
+    config_subtest "dump" "$config"
     notmuch dump is:attachment and is:signed | sort > OUTPUT
     cat <<EOF > EXPECTED
 #notmuch-dump batch-tag:3 config,properties,tags
@@ -136,43 +146,43 @@ EOF
 EOF
     test_expect_equal_file EXPECTED OUTPUT
 
-    test_begin_subtest "dump + tag + restore ($config)"
+    config_subtest "dump + tag + restore" "$config"
     notmuch dump '*' > EXPECTED
     notmuch tag -inbox '*'
     notmuch restore < EXPECTED
     notmuch dump > OUTPUT
     test_expect_equal_file EXPECTED OUTPUT
 
-    test_begin_subtest "reindex ($config)"
+    config_subtest "reindex" "$config"
     notmuch search --output=messages '*' > EXPECTED
     notmuch reindex '*'
     notmuch search --output=messages '*' > OUTPUT
     test_expect_equal_file EXPECTED OUTPUT
 
-    test_begin_subtest "use existing database ($config)"
+    config_subtest "use existing database" "$config"
     output=$(notmuch new)
     test_expect_equal "$output" 'No new mail.'
 
-    test_begin_subtest "create database ($config)"
+    config_subtest "create database" "$config"
     rm -rf $DATABASE_PATH/{.notmuch,}/xapian
     notmuch new
     output=$(notmuch count '*')
     test_expect_equal "$output" '52'
 
-    test_begin_subtest "detect new files ($config)"
+    config_subtest "detect new files" "$config"
     generate_message
     generate_message
     notmuch new
     output=$(notmuch count '*')
     test_expect_equal "$output" '54'
 
-    test_begin_subtest "Show a raw message ($config)"
+    config_subtest "Show a raw message" "$config"
     add_message
     notmuch show --format=raw id:$gen_msg_id > OUTPUT
     test_expect_equal_file $gen_msg_filename OUTPUT
     rm -f $gen_msg_filename
 
-    test_begin_subtest "reply ($config)"
+    config_subtest "reply" "$config"
     add_message '[from]="Sender <sender@example.com>"' \
 		[to]=test_suite@notmuchmail.org \
 		[subject]=notmuch-reply-test \
@@ -190,7 +200,7 @@ On Tue, 05 Jan 2010 15:43:56 -0000, Sender <sender@example.com> wrote:
 > basic reply test
 EOF
     test_expect_equal_file EXPECTED OUTPUT
-    test_begin_subtest "insert+search ($config)"
+    config_subtest "insert+search" "$config"
     generate_message \
 	"[subject]=\"insert-subject\"" \
 	"[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" \
@@ -201,13 +211,13 @@ EOF
     test_expect_equal_file "$cur_msg_filename" "$gen_msg_filename"
 
 
-    test_begin_subtest "compact+search ($config)"
+    config_subtest "compact+search" "$config"
     notmuch search --output=messages '*' | sort > EXPECTED
     notmuch compact
     notmuch search --output=messages '*' | sort > OUTPUT
     test_expect_equal_file EXPECTED OUTPUT
 
-    test_begin_subtest "upgrade backup ($config)"
+    config_subtest "upgrade backup" "$config"
     features=$(xapian-metadata get $XAPIAN_PATH features | grep -v "^relative directory paths")
     xapian-metadata set $XAPIAN_PATH features "$features"
     output=$(notmuch new | grep Welcome)
@@ -215,7 +225,7 @@ EOF
 	"$output" \
 	"Welcome to a new version of notmuch! Your database will now be upgraded."
 
-    test_begin_subtest "notmuch +config -database suggests notmuch new ($config)"
+    config_subtest "notmuch +config -database suggests notmuch new" "$config"
     mv "$XAPIAN_PATH" "${XAPIAN_PATH}.bak"
     notmuch > OUTPUT
 cat <<EOF > EXPECTED
@@ -231,7 +241,7 @@ EOF
 
    test_expect_equal_file EXPECTED OUTPUT
 
-   test_begin_subtest "Set config value ($config)"
+   config_subtest "Set config value" "$config"
    name=${RANDOM}
    value=${RANDOM}
    notmuch config set test${test_count}.${name} ${value}
@@ -240,7 +250,7 @@ EOF
    output2=$(notmuch config get test${test_count}.${name})
    test_expect_equal "${output}+${output2}" "${value}+"
 
-   test_begin_subtest "Set config value in database ($config)"
+   config_subtest "Set config value in database" "$config"
    name=${RANDOM}
    value=${RANDOM}
    notmuch config set --database test${test_count}.${name} ${value}
@@ -249,7 +259,7 @@ EOF
    output2=$(notmuch config get test${test_count}.${name})
    test_expect_equal "${output}+${output2}" "${value}+"
 
-   test_begin_subtest "Config list ($config)"
+   config_subtest "Config list" "$config"
    notmuch config list | notmuch_dir_sanitize | sed -e "s/^database.backup_dir=.*$/database.backup_dir/"  \
 						    -e "s/^database.hook_dir=.*$/database.hook_dir/" \
 						    -e "s/^database.path=.*$/database.path/"  \
@@ -276,7 +286,7 @@ EOF
 
    case $config in
        XDG*)
-	   test_begin_subtest "Set shadowed config value in database ($config)"
+	   config_subtest "Set shadowed config value in database" "$config"
 	   name=${RANDOM}
 	   value=${RANDOM}
 	   key=test${test_count}.${name}
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/8] test/expect_equal_file: whitespace cleanup
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
  2021-08-02 15:09   ` [PATCH 1/8] test/path-config: define config_subtest David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-02 15:09   ` [PATCH 3/8] test: factor out test_diff_file_ David Bremner
                     ` (5 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

No functional change, just reindent to Emacs defaults.
---
 test/test-lib.sh | 42 +++++++++++++++++++++---------------------
 1 file changed, 21 insertions(+), 21 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 67ad8853..10ee9b72 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -404,29 +404,29 @@ test_expect_equal () {
 
 # Like test_expect_equal, but takes two filenames.
 test_expect_equal_file () {
-	local file1 file2 testname basename1 basename2
-	exec 1>&6 2>&7		# Restore stdout and stderr
-	if [ -z "$inside_subtest" ]; then
-		error "bug in the test script: test_expect_equal_file without test_begin_subtest"
-	fi
-	inside_subtest=
-	test "$#" = 2 ||
+    local file1 file2 testname basename1 basename2
+    exec 1>&6 2>&7		# Restore stdout and stderr
+    if [ -z "$inside_subtest" ]; then
+	error "bug in the test script: test_expect_equal_file without test_begin_subtest"
+    fi
+    inside_subtest=
+    test "$#" = 2 ||
 	error "bug in the test script: not 2 parameters to test_expect_equal_file"
 
-	file1="$1"
-	file2="$2"
-	if ! test_skip "$test_subtest_name"
-	then
-		if diff -q "$file1" "$file2" >/dev/null ; then
-			test_ok_
-		else
-			testname=$this_test.$test_count
-			basename1=`basename "$file1"`
-			basename2=`basename "$file2"`
-			cp "$file1" "$testname.$basename1"
-			cp "$file2" "$testname.$basename2"
-			test_failure_ "$(diff -u "$testname.$basename1" "$testname.$basename2")"
-		fi
+    file1="$1"
+    file2="$2"
+    if ! test_skip "$test_subtest_name"
+    then
+	if diff -q "$file1" "$file2" >/dev/null ; then
+	    test_ok_
+	else
+	    testname=$this_test.$test_count
+	    basename1=`basename "$file1"`
+	    basename2=`basename "$file2"`
+	    cp "$file1" "$testname.$basename1"
+	    cp "$file2" "$testname.$basename2"
+	    test_failure_ "$(diff -u "$testname.$basename1" "$testname.$basename2")"
+	fi
     fi
 }
 
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/8] test: factor out test_diff_file_
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
  2021-08-02 15:09   ` [PATCH 1/8] test/path-config: define config_subtest David Bremner
  2021-08-02 15:09   ` [PATCH 2/8] test/expect_equal_file: whitespace cleanup David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-02 15:09   ` [PATCH 4/8] test: add test_expect_equal_file_nonempty David Bremner
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

A following commit will use this new function in a public test
function.
---
 test/test-lib.sh | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 10ee9b72..a8e57e51 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -402,17 +402,8 @@ test_expect_equal () {
     fi
 }
 
-# Like test_expect_equal, but takes two filenames.
-test_expect_equal_file () {
+test_diff_file_ () {
     local file1 file2 testname basename1 basename2
-    exec 1>&6 2>&7		# Restore stdout and stderr
-    if [ -z "$inside_subtest" ]; then
-	error "bug in the test script: test_expect_equal_file without test_begin_subtest"
-    fi
-    inside_subtest=
-    test "$#" = 2 ||
-	error "bug in the test script: not 2 parameters to test_expect_equal_file"
-
     file1="$1"
     file2="$2"
     if ! test_skip "$test_subtest_name"
@@ -430,6 +421,19 @@ test_expect_equal_file () {
     fi
 }
 
+# Like test_expect_equal, but takes two filenames.
+test_expect_equal_file () {
+    exec 1>&6 2>&7		# Restore stdout and stderr
+    if [ -z "$inside_subtest" ]; then
+	error "bug in the test script: test_expect_equal_file without test_begin_subtest"
+    fi
+    inside_subtest=
+    test "$#" = 2 ||
+	error "bug in the test script: not 2 parameters to test_expect_equal_file"
+
+    test_diff_file_ "$1" "$2"
+}
+
 # Like test_expect_equal, but arguments are JSON expressions to be
 # canonicalized before diff'ing.  If an argument cannot be parsed, it
 # is used unchanged so that there's something to diff against.
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 4/8] test: add test_expect_equal_file_nonempty
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
                     ` (2 preceding siblings ...)
  2021-08-02 15:09   ` [PATCH 3/8] test: factor out test_diff_file_ David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-02 15:09   ` [PATCH 5/8] test/path-config: use test_expect_equal_file_nonempty David Bremner
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

A common bug in tests is that the code used to generate the EXPECTED
file fails, generating no output. When the code generating the OUTPUT
file fails in the same way, the test passes, even though there is a
failure being hidden. Add a new test function that guards against
this.
---
 test/test-lib.sh | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index a8e57e51..4865032d 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -434,6 +434,25 @@ test_expect_equal_file () {
     test_diff_file_ "$1" "$2"
 }
 
+# Like test_expect_equal, but takes two filenames. Fails if either is empty
+test_expect_equal_file_nonempty () {
+    exec 1>&6 2>&7		# Restore stdout and stderr
+    if [ -z "$inside_subtest" ]; then
+	error "bug in the test script: test_expect_equal_file_nonempty without test_begin_subtest"
+    fi
+    inside_subtest=
+    test "$#" = 2 ||
+	error "bug in the test script: not 2 parameters to test_expect_equal_file_nonempty"
+
+    for file in "$1" "$2"; do
+	if [ ! -s "$file" ]; then
+	    test_failure_ "Missing or zero length file: $file"
+	fi
+    done
+
+    test_diff_file_ "$1" "$2"
+}
+
 # Like test_expect_equal, but arguments are JSON expressions to be
 # canonicalized before diff'ing.  If an argument cannot be parsed, it
 # is used unchanged so that there's something to diff against.
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 5/8] test/path-config: use test_expect_equal_file_nonempty
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
                     ` (3 preceding siblings ...)
  2021-08-02 15:09   ` [PATCH 4/8] test: add test_expect_equal_file_nonempty David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-02 15:09   ` [PATCH 6/8] test: add known broken tests for finding database via MAILDIR David Bremner
                     ` (2 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This is more robust against crashes when the expected output is also
generated by notmuch. In the case where the expected output is
explicit, it seems like overkill.
---
 test/T055-path-config.sh | 11 +++++------
 test/test-lib.sh         |  1 +
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh
index 2061b116..035276c2 100755
--- a/test/T055-path-config.sh
+++ b/test/T055-path-config.sh
@@ -151,13 +151,13 @@ EOF
     notmuch tag -inbox '*'
     notmuch restore < EXPECTED
     notmuch dump > OUTPUT
-    test_expect_equal_file EXPECTED OUTPUT
+    test_expect_equal_file_nonempty EXPECTED OUTPUT
 
     config_subtest "reindex" "$config"
     notmuch search --output=messages '*' > EXPECTED
     notmuch reindex '*'
     notmuch search --output=messages '*' > OUTPUT
-    test_expect_equal_file EXPECTED OUTPUT
+    test_expect_equal_file_nonempty EXPECTED OUTPUT
 
     config_subtest "use existing database" "$config"
     output=$(notmuch new)
@@ -179,7 +179,7 @@ EOF
     config_subtest "Show a raw message" "$config"
     add_message
     notmuch show --format=raw id:$gen_msg_id > OUTPUT
-    test_expect_equal_file $gen_msg_filename OUTPUT
+    test_expect_equal_file_nonempty $gen_msg_filename OUTPUT
     rm -f $gen_msg_filename
 
     config_subtest "reply" "$config"
@@ -208,14 +208,13 @@ EOF
     mkdir -p "$MAIL_DIR"/{cur,new,tmp}
     notmuch insert < "$gen_msg_filename"
     cur_msg_filename=$(notmuch search --output=files "subject:insert-subject")
-    test_expect_equal_file "$cur_msg_filename" "$gen_msg_filename"
-
+    test_expect_equal_file_nonempty "$cur_msg_filename" "$gen_msg_filename"
 
     config_subtest "compact+search" "$config"
     notmuch search --output=messages '*' | sort > EXPECTED
     notmuch compact
     notmuch search --output=messages '*' | sort > OUTPUT
-    test_expect_equal_file EXPECTED OUTPUT
+    test_expect_equal_file_nonempty EXPECTED OUTPUT
 
     config_subtest "upgrade backup" "$config"
     features=$(xapian-metadata get $XAPIAN_PATH features | grep -v "^relative directory paths")
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 4865032d..94f9d605 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -447,6 +447,7 @@ test_expect_equal_file_nonempty () {
     for file in "$1" "$2"; do
 	if [ ! -s "$file" ]; then
 	    test_failure_ "Missing or zero length file: $file"
+	    return $?
 	fi
     done
 
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 6/8] test: add known broken tests for finding database via MAILDIR
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
                     ` (4 preceding siblings ...)
  2021-08-02 15:09   ` [PATCH 5/8] test/path-config: use test_expect_equal_file_nonempty David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-02 15:09   ` [PATCH 7/8] lib/open: look in MAILDIR for database, as documented David Bremner
  2021-08-02 15:09   ` [PATCH 8/8] doc: document database search algorithm David Bremner
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This highlights a bug reported by several users, including
Mohsin Kaleem [1].

The use of test_begin_subtest is because these tests pass even though
the database cannot be located.

[1]: id:87bl9lx864.fsf@kisara.moe
---
 test/T055-path-config.sh | 34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)

diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh
index 035276c2..40601acd 100755
--- a/test/T055-path-config.sh
+++ b/test/T055-path-config.sh
@@ -16,6 +16,7 @@ restore_config () {
     unset DATABASE_PATH
     unset NOTMUCH_PROFILE
     unset XAPIAN_PATH
+    unset MAILDIR
     rm -f "$HOME/mail"
     cp notmuch-config-backup.${test_name} ${NOTMUCH_CONFIG}
 }
@@ -55,6 +56,18 @@ home_mail_config () {
     unset DATABASE_PATH
 }
 
+maildir_env_config () {
+    local dir
+    backup_config
+    dir="${HOME}/env_points_here"
+    ln -s $MAIL_DIR $dir
+    export MAILDIR=$dir
+    notmuch config set database.path
+    notmuch config set database.mail_root
+    XAPIAN_PATH="${MAIL_DIR}/.notmuch/xapian"
+    unset DATABASE_PATH
+}
+
 xdg_config () {
     local dir
     local profile=${1:-default}
@@ -84,9 +97,12 @@ config_subtest () {
     local config=$2
 
     test_begin_subtest "$title ($config)"
+    if [[ "$config" == "maildir_env" ]]; then
+	test_subtest_known_broken
+    fi
 }
 
-for config in traditional split XDG XDG+profile symlink home_mail; do
+for config in traditional split XDG XDG+profile symlink home_mail maildir_env; do
     #start each set of tests with an known set of messages
     add_email_corpus
 
@@ -224,7 +240,7 @@ EOF
 	"$output" \
 	"Welcome to a new version of notmuch! Your database will now be upgraded."
 
-    config_subtest "notmuch +config -database suggests notmuch new" "$config"
+    test_begin_subtest "notmuch +config -database suggests notmuch new" "$config"
     mv "$XAPIAN_PATH" "${XAPIAN_PATH}.bak"
     notmuch > OUTPUT
 cat <<EOF > EXPECTED
@@ -240,7 +256,7 @@ EOF
 
    test_expect_equal_file EXPECTED OUTPUT
 
-   config_subtest "Set config value" "$config"
+   test_begin_subtest "Set config value" "$config"
    name=${RANDOM}
    value=${RANDOM}
    notmuch config set test${test_count}.${name} ${value}
@@ -259,11 +275,13 @@ EOF
    test_expect_equal "${output}+${output2}" "${value}+"
 
    config_subtest "Config list" "$config"
-   notmuch config list | notmuch_dir_sanitize | sed -e "s/^database.backup_dir=.*$/database.backup_dir/"  \
-						    -e "s/^database.hook_dir=.*$/database.hook_dir/" \
-						    -e "s/^database.path=.*$/database.path/"  \
-						    -e "s,^database.mail_root=CWD/home/mail,database.mail_root=MAIL_DIR," \
-						    > OUTPUT
+   notmuch config list | notmuch_dir_sanitize | \
+       sed -e "s/^database.backup_dir=.*$/database.backup_dir/"  \
+	   -e "s/^database.hook_dir=.*$/database.hook_dir/" \
+	   -e "s/^database.path=.*$/database.path/"  \
+	   -e "s,^database.mail_root=CWD/home/mail,database.mail_root=MAIL_DIR," \
+	   -e "s,^database.mail_root=CWD/home/env_points_here,database.mail_root=MAIL_DIR," \
+	   > OUTPUT
    cat <<EOF > EXPECTED
 built_with.compact=true
 built_with.field_processor=true
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 7/8] lib/open: look in MAILDIR for database, as documented.
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
                     ` (5 preceding siblings ...)
  2021-08-02 15:09   ` [PATCH 6/8] test: add known broken tests for finding database via MAILDIR David Bremner
@ 2021-08-02 15:09   ` David Bremner
  2021-08-02 15:09   ` [PATCH 8/8] doc: document database search algorithm David Bremner
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

This fixes the bug id:87bl9lx864.fsf@kisara.moe
---
 lib/open.cc              | 4 ++++
 test/T055-path-config.sh | 3 ---
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/lib/open.cc b/lib/open.cc
index 7b95c5b1..280ffee3 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -220,6 +220,10 @@ _choose_database_path (void *ctx,
 	}
     }
 
+    if (! *database_path) {
+	*database_path = getenv ("MAILDIR");
+    }
+
     if (! *database_path) {
 	notmuch_status_t status;
 
diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh
index 40601acd..774fe9c8 100755
--- a/test/T055-path-config.sh
+++ b/test/T055-path-config.sh
@@ -97,9 +97,6 @@ config_subtest () {
     local config=$2
 
     test_begin_subtest "$title ($config)"
-    if [[ "$config" == "maildir_env" ]]; then
-	test_subtest_known_broken
-    fi
 }
 
 for config in traditional split XDG XDG+profile symlink home_mail maildir_env; do
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 8/8] doc: document database search algorithm.
  2021-08-02 15:09 ` look in MAILDIR for database David Bremner
                     ` (6 preceding siblings ...)
  2021-08-02 15:09   ` [PATCH 7/8] lib/open: look in MAILDIR for database, as documented David Bremner
@ 2021-08-02 15:09   ` David Bremner
  7 siblings, 0 replies; 11+ messages in thread
From: David Bremner @ 2021-08-02 15:09 UTC (permalink / raw)
  To: notmuch; +Cc: David Bremner

Essentially a translation of the function _choose_database_path for
human consumption. As a bonus, document environment variable
NOTMUCH_DATABASE
---
 doc/man1/notmuch-config.rst | 21 ++++++++++++++++++++-
 doc/man1/notmuch.rst        |  5 +++++
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 82fbb44c..07a9eaf0 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -60,7 +60,7 @@ paths are presumed relative to `$HOME` for items in section
     sub-directory named ``.notmuch`` if **database.mail\_root**
     is unset).
 
-    Default: ``$MAILDIR`` variable if set, otherwise ``$HOME/mail``.
+    Default: see :ref:`database`
 
 **database.mail_root**
     The top-level directory where your mail currently exists and to
@@ -272,6 +272,25 @@ Notmuch configuration file search order:
    by :envvar:`NOTMUCH_PROFILE` environment variable if set,
    ``$HOME/.notmuch-config`` otherwise.
 
+.. _database:
+
+DATABASE LOCATION
+-----------------
+
+Notmuch database search order:
+
+1. Directory specified by :envvar:`NOTMUCH_DATABASE` environment variable.
+
+2. Directory specified by config key ``database.path``.
+
+3. ``$XDG_DATA_HOME/notmuch/<profile>`` where ``<profile>``
+   is defined by :envvar:`NOTMUCH_PROFILE` environment variable if
+   set, ``$XDG_DATA_HOME/notmuch/default`` otherwise.
+
+4. Directory specified by :envvar:`MAILDIR` environment variable.
+
+5. ``$HOME/mail``
+
 HOOKS
 -----
 
diff --git a/doc/man1/notmuch.rst b/doc/man1/notmuch.rst
index 6bd1dc03..de4c91ac 100644
--- a/doc/man1/notmuch.rst
+++ b/doc/man1/notmuch.rst
@@ -164,6 +164,11 @@ of notmuch.
    Specifies the location of the notmuch configuration file. See
    :any:`notmuch-config(1)` for details.
 
+.. envvar:: NOTMUCH_DATABASE
+
+   Specifies the location of the notmuch database. See
+   :any:`notmuch-config(1)` for details.
+        
 .. envvar:: NOTMUCH_PROFILE
 
    Selects among notmuch configurations. See :any:`notmuch-config(1)`
-- 
2.30.2

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH 1/8] test/path-config: define config_subtest
  2021-08-02 15:09   ` [PATCH 1/8] test/path-config: define config_subtest David Bremner
@ 2021-08-03 20:26     ` Tomi Ollila
  0 siblings, 0 replies; 11+ messages in thread
From: Tomi Ollila @ 2021-08-03 20:26 UTC (permalink / raw)
  To: David Bremner, notmuch

On Mon, Aug 02 2021, David Bremner wrote:

> This will allow adding "hook code" to be run for a particular
> configuration, in particular marking all tests in a given
> configuration as broken.

uh, see below

> ---
>  test/T055-path-config.sh | 48 ++++++++++++++++++++++++----------------
>  1 file changed, 29 insertions(+), 19 deletions(-)
>
> diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh
> index bb3bf665..2061b116 100755
> --- a/test/T055-path-config.sh
> +++ b/test/T055-path-config.sh
> @@ -79,6 +79,13 @@ xdg_config () {
>      notmuch --config=${CONFIG_PATH} config set database.path
>  }
>  
> +config_subtest () {
> +    local title=$1
> +    local config=$2
> +
> +    test_begin_subtest "$title ($config)"
> +}

here w/o commit message one cannot see the point ;/

Then I looked, in one commit there were this added

      if [[ "$config" == "maildir_env" ]]; then
  	test_subtest_known_broken
      fi

after test_begin_subtest "$title ($config)", and one
config_subtest call was restored to call to test_begin_subtest

finally the above code was removed (but that one test_begin_subtest
was not re-restored to use config_subtest)

i am not too happy about this acrobacy...

btw; that one change there (later in series):

  -    config_subtest "notmuch +config -database suggests notmuch new" "$config"
  +    test_begin_subtest "notmuch +config -database suggests notmuch new" "$config"

... and another...

  -   config_subtest "Set config value" "$config"
  +   test_begin_subtest "Set config value" "$config"

are these correct, the latter was changed originally (in this patch):

  -   test_begin_subtest "Set config value ($config)"
  +   config_subtest "Set config value" "$config"

---

After thinking a while I see the point, I am not sure if this is better,
but I'd do this the following way:

begin_config_subtest () {
    test_begin_subtest "$@"
}

then it is easy to flip between begin_config_subtest and test_begin_subtest
whenever needed -- and in final commit to fix all broken patches remove
all extra code from begin_config_subtest and ensure that no calls to
test_begin_subtest are made in that particular file


Tomi


> +
>  for config in traditional split XDG XDG+profile symlink home_mail; do
>      #start each set of tests with an known set of messages
>      add_email_corpus
> @@ -106,20 +113,23 @@ for config in traditional split XDG XDG+profile symlink home_mail; do
>  	home_mail)
>  	    home_mail_config
>  	    ;;
> +	maildir_env)
> +	    maildir_env_config
> +	    ;;
>      esac
>  
> -    test_begin_subtest "count ($config)"
> +    config_subtest "count" "$config"
>      output=$(notmuch count '*')
>      test_expect_equal "$output" '52'
>  
> -    test_begin_subtest "count+tag ($config)"
> +    config_subtest "count+tag" "$config"
>      tag="tag${RANDOM}"
>      notmuch tag +$tag '*'
>      output=$(notmuch count tag:$tag)
>      notmuch tag -$tag '*'
>      test_expect_equal "$output" '52'
>  
> -    test_begin_subtest "address ($config)"
> +    config_subtest "address" "$config"
>      notmuch address --deduplicate=no --sort=newest-first --output=sender --output=recipients path:foo >OUTPUT
>      cat <<EOF >EXPECTED
>  Carl Worth <cworth@cworth.org>
> @@ -127,7 +137,7 @@ notmuch@notmuchmail.org
>  EOF
>      test_expect_equal_file EXPECTED OUTPUT
>  
> -    test_begin_subtest "dump ($config)"
> +    config_subtest "dump" "$config"
>      notmuch dump is:attachment and is:signed | sort > OUTPUT
>      cat <<EOF > EXPECTED
>  #notmuch-dump batch-tag:3 config,properties,tags
> @@ -136,43 +146,43 @@ EOF
>  EOF
>      test_expect_equal_file EXPECTED OUTPUT
>  
> -    test_begin_subtest "dump + tag + restore ($config)"
> +    config_subtest "dump + tag + restore" "$config"
>      notmuch dump '*' > EXPECTED
>      notmuch tag -inbox '*'
>      notmuch restore < EXPECTED
>      notmuch dump > OUTPUT
>      test_expect_equal_file EXPECTED OUTPUT
>  
> -    test_begin_subtest "reindex ($config)"
> +    config_subtest "reindex" "$config"
>      notmuch search --output=messages '*' > EXPECTED
>      notmuch reindex '*'
>      notmuch search --output=messages '*' > OUTPUT
>      test_expect_equal_file EXPECTED OUTPUT
>  
> -    test_begin_subtest "use existing database ($config)"
> +    config_subtest "use existing database" "$config"
>      output=$(notmuch new)
>      test_expect_equal "$output" 'No new mail.'
>  
> -    test_begin_subtest "create database ($config)"
> +    config_subtest "create database" "$config"
>      rm -rf $DATABASE_PATH/{.notmuch,}/xapian
>      notmuch new
>      output=$(notmuch count '*')
>      test_expect_equal "$output" '52'
>  
> -    test_begin_subtest "detect new files ($config)"
> +    config_subtest "detect new files" "$config"
>      generate_message
>      generate_message
>      notmuch new
>      output=$(notmuch count '*')
>      test_expect_equal "$output" '54'
>  
> -    test_begin_subtest "Show a raw message ($config)"
> +    config_subtest "Show a raw message" "$config"
>      add_message
>      notmuch show --format=raw id:$gen_msg_id > OUTPUT
>      test_expect_equal_file $gen_msg_filename OUTPUT
>      rm -f $gen_msg_filename
>  
> -    test_begin_subtest "reply ($config)"
> +    config_subtest "reply" "$config"
>      add_message '[from]="Sender <sender@example.com>"' \
>  		[to]=test_suite@notmuchmail.org \
>  		[subject]=notmuch-reply-test \
> @@ -190,7 +200,7 @@ On Tue, 05 Jan 2010 15:43:56 -0000, Sender <sender@example.com> wrote:
>  > basic reply test
>  EOF
>      test_expect_equal_file EXPECTED OUTPUT
> -    test_begin_subtest "insert+search ($config)"
> +    config_subtest "insert+search" "$config"
>      generate_message \
>  	"[subject]=\"insert-subject\"" \
>  	"[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" \
> @@ -201,13 +211,13 @@ EOF
>      test_expect_equal_file "$cur_msg_filename" "$gen_msg_filename"
>  
>  
> -    test_begin_subtest "compact+search ($config)"
> +    config_subtest "compact+search" "$config"
>      notmuch search --output=messages '*' | sort > EXPECTED
>      notmuch compact
>      notmuch search --output=messages '*' | sort > OUTPUT
>      test_expect_equal_file EXPECTED OUTPUT
>  
> -    test_begin_subtest "upgrade backup ($config)"
> +    config_subtest "upgrade backup" "$config"
>      features=$(xapian-metadata get $XAPIAN_PATH features | grep -v "^relative directory paths")
>      xapian-metadata set $XAPIAN_PATH features "$features"
>      output=$(notmuch new | grep Welcome)
> @@ -215,7 +225,7 @@ EOF
>  	"$output" \
>  	"Welcome to a new version of notmuch! Your database will now be upgraded."
>  
> -    test_begin_subtest "notmuch +config -database suggests notmuch new ($config)"
> +    config_subtest "notmuch +config -database suggests notmuch new" "$config"
>      mv "$XAPIAN_PATH" "${XAPIAN_PATH}.bak"
>      notmuch > OUTPUT
>  cat <<EOF > EXPECTED
> @@ -231,7 +241,7 @@ EOF
>  
>     test_expect_equal_file EXPECTED OUTPUT
>  
> -   test_begin_subtest "Set config value ($config)"
> +   config_subtest "Set config value" "$config"
>     name=${RANDOM}
>     value=${RANDOM}
>     notmuch config set test${test_count}.${name} ${value}
> @@ -240,7 +250,7 @@ EOF
>     output2=$(notmuch config get test${test_count}.${name})
>     test_expect_equal "${output}+${output2}" "${value}+"
>  
> -   test_begin_subtest "Set config value in database ($config)"
> +   config_subtest "Set config value in database" "$config"
>     name=${RANDOM}
>     value=${RANDOM}
>     notmuch config set --database test${test_count}.${name} ${value}
> @@ -249,7 +259,7 @@ EOF
>     output2=$(notmuch config get test${test_count}.${name})
>     test_expect_equal "${output}+${output2}" "${value}+"
>  
> -   test_begin_subtest "Config list ($config)"
> +   config_subtest "Config list" "$config"
>     notmuch config list | notmuch_dir_sanitize | sed -e "s/^database.backup_dir=.*$/database.backup_dir/"  \
>  						    -e "s/^database.hook_dir=.*$/database.hook_dir/" \
>  						    -e "s/^database.path=.*$/database.path/"  \
> @@ -276,7 +286,7 @@ EOF
>  
>     case $config in
>         XDG*)
> -	   test_begin_subtest "Set shadowed config value in database ($config)"
> +	   config_subtest "Set shadowed config value in database" "$config"
>  	   name=${RANDOM}
>  	   value=${RANDOM}
>  	   key=test${test_count}.${name}
> -- 
> 2.30.2
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-08-03 20:26 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-08 20:49 [kisara.moe] notmuch doesn't respect MAILDIR env for database path mohsin kaleem
2021-08-02 15:09 ` look in MAILDIR for database David Bremner
2021-08-02 15:09   ` [PATCH 1/8] test/path-config: define config_subtest David Bremner
2021-08-03 20:26     ` Tomi Ollila
2021-08-02 15:09   ` [PATCH 2/8] test/expect_equal_file: whitespace cleanup David Bremner
2021-08-02 15:09   ` [PATCH 3/8] test: factor out test_diff_file_ David Bremner
2021-08-02 15:09   ` [PATCH 4/8] test: add test_expect_equal_file_nonempty David Bremner
2021-08-02 15:09   ` [PATCH 5/8] test/path-config: use test_expect_equal_file_nonempty David Bremner
2021-08-02 15:09   ` [PATCH 6/8] test: add known broken tests for finding database via MAILDIR David Bremner
2021-08-02 15:09   ` [PATCH 7/8] lib/open: look in MAILDIR for database, as documented David Bremner
2021-08-02 15:09   ` [PATCH 8/8] doc: document database search algorithm David Bremner

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).