From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id iFJVOWsNRWAiUwAA0tVLHw (envelope-from ) for ; Sun, 07 Mar 2021 17:29:15 +0000 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id uGUqNWsNRWAiVgAA1q6Kng (envelope-from ) for ; Sun, 07 Mar 2021 17:29:15 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 29154149E1 for ; Sun, 7 Mar 2021 18:29:15 +0100 (CET) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id EED0727C5A; Sun, 7 Mar 2021 12:28:22 -0500 (EST) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 4018F27C98 for ; Sun, 7 Mar 2021 12:27:56 -0500 (EST) Received: by fethera.tethera.net (Postfix, from userid 1001) id 379775FA96; Sun, 7 Mar 2021 12:27:56 -0500 (EST) Received: (nullmailer pid 1091662 invoked by uid 1000); Sun, 07 Mar 2021 17:27:38 -0000 From: David Bremner To: notmuch@notmuchmail.org Cc: David Bremner Subject: [PATCH 14/22] support splitting mail from database location. Date: Sun, 7 Mar 2021 13:27:26 -0400 Message-Id: <20210307172734.1091264-15-david@tethera.net> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210307172734.1091264-1-david@tethera.net> References: <20210307172734.1091264-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: 3YK4G5VVARAHRH6A3LF4WV4MUAJ76ZGS X-Message-ID-Hash: 3YK4G5VVARAHRH6A3LF4WV4MUAJ76ZGS X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615138155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ucoaMbaSN2pWXMMGpOPkhyNG7V7XVqPOrr4HeCQivrI=; b=rDkF5pCaTanrdHna4jO/0sdsPIA2aQzIZVA04BzgmjjlgTtRjDejF2EehEB2hYbCZbOJps 8kXLBDbgbXpiiFWuIHNf0Dw5s7KQV+Ci+u+Geo/4RKY/G2HlaVuFha+ByjFfY7eHXbufON cvS1U/O4zlRWNz+GuSsktZD+Aso2psYnGZ/Wuu5cHXiUCle6pkdk1kKpoxViYwLCcyE5du XRw3Ov8AXgcR1S0jZXdfKgv2qtD1vGcy06vDlKjycPRPyfAp09Vi0FUrx+MHshmt5Tu8M0 qmsZjs4ICBofx2lfTIsh3c1wK0PamfoDOfQWpKXVXHQsEnpGZqstkoQUxY7ekA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615138155; a=rsa-sha256; cv=none; b=jUSfj5gIGbnx23jgidjvDybL3EKxTU7kiqWsaCOwci39nS7maXpyISXaJWTN+DgAUr75l9 cBPndGFBLs59NzyVby3z3nYpBdUvQ95YxAFzXMiE1NYqwIhW3shA0YwBCRSjFPJRwUuQfg ulpGVKssmbEXYn3SDWGQt9cUlp0XKA4tsF0jlW5OhFusDEZE7aFQr0E80kPyWM9De+sm4F Uf6Wj5NetFKBeJXi3RVUaK7QLQD0J2BcRwgjyGUtCf+g7b/jm547mzzk4BqQWQiJULd/jB gqzILiehb1719nsknc0rPrr3lDUH9J/0yhzZu13JZvu8xePrPTBrm9S7acDBDQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -0.74 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 29154149E1 X-Spam-Score: -0.74 X-Migadu-Scanner: scn1.migadu.com X-TUID: /Hb6XmxL4gAy Introduce a new configuration value for the mail root, and use it in preference to the database.path (which previously implied the mail was also in this location. Further changes to the CLI will be needed to work in this split configuration. --- doc/man1/notmuch-config.rst | 14 ++++-- lib/config.cc | 11 +++-- lib/database.cc | 2 +- lib/message-file.c | 2 +- lib/message.cc | 2 +- lib/notmuch.h | 1 + lib/open.cc | 5 +++ test/T055-path-config.sh | 90 +++++++++++++++++++++++++++++++++++++ test/T590-libconfig.sh | 1 + 9 files changed, 119 insertions(+), 9 deletions(-) create mode 100755 test/T055-path-config.sh diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst index bc597957..99030a41 100644 --- a/doc/man1/notmuch-config.rst +++ b/doc/man1/notmuch-config.rst @@ -43,12 +43,20 @@ configuration file and corresponding database. The available configuration items are described below. **database.path** + Notmuch will store its database within a + sub-directory of the path configured here named ``.notmuch``. + + Default: ``$MAILDIR`` variable if set, otherwise ``$HOME/mail``. + +**database.mail_root** The top-level directory where your mail currently exists and to where mail will be delivered in the future. Files should be - individual email messages. Notmuch will store its database within - a sub-directory of the path configured here named ``.notmuch``. + individual email messages. - Default: ``$MAILDIR`` variable if set, otherwise ``$HOME/mail``. + History: this configuration value was introduced in notmuch 0.32. + + Default: For compatibility with older configurations, the value of + database.path is used if database.mail\_root is unset. **database.hook_dir** diff --git a/lib/config.cc b/lib/config.cc index 64bcf692..04b81808 100644 --- a/lib/config.cc +++ b/lib/config.cc @@ -405,6 +405,8 @@ _notmuch_config_key_to_string (notmuch_config_key_t key) switch (key) { case NOTMUCH_CONFIG_DATABASE_PATH: return "database.path"; + case NOTMUCH_CONFIG_MAIL_ROOT: + return "database.mail_root"; case NOTMUCH_CONFIG_HOOK_DIR: return "database.hook_dir"; case NOTMUCH_CONFIG_EXCLUDE_TAGS: @@ -427,7 +429,7 @@ _notmuch_config_key_to_string (notmuch_config_key_t key) } static const char * -_notmuch_config_default (void *ctx, notmuch_config_key_t key) +_notmuch_config_default (notmuch_database_t *notmuch, notmuch_config_key_t key) { char *path; @@ -435,11 +437,14 @@ _notmuch_config_default (void *ctx, notmuch_config_key_t key) case NOTMUCH_CONFIG_DATABASE_PATH: path = getenv ("MAILDIR"); if (path) - path = talloc_strdup (ctx, path); + path = talloc_strdup (notmuch, path); else - path = talloc_asprintf (ctx, "%s/mail", + path = talloc_asprintf (notmuch, "%s/mail", getenv ("HOME")); return path; + case NOTMUCH_CONFIG_MAIL_ROOT: + /* by default, mail root is the same as database path */ + return notmuch_database_get_path (notmuch); case NOTMUCH_CONFIG_EXCLUDE_TAGS: return ""; case NOTMUCH_CONFIG_NEW_TAGS: diff --git a/lib/database.cc b/lib/database.cc index 155232f9..12ab1af7 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -1347,7 +1347,7 @@ _notmuch_database_relative_path (notmuch_database_t *notmuch, const char *db_path, *relative; unsigned int db_path_len; - db_path = notmuch_database_get_path (notmuch); + db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_MAIL_ROOT); db_path_len = strlen (db_path); relative = path; diff --git a/lib/message-file.c b/lib/message-file.c index 311bd478..a23493f1 100644 --- a/lib/message-file.c +++ b/lib/message-file.c @@ -64,7 +64,7 @@ _notmuch_message_file_open_ctx (notmuch_database_t *notmuch, if (unlikely (message == NULL)) return NULL; - const char *prefix = notmuch_database_get_path (notmuch); + const char *prefix = notmuch_config_get (notmuch, NOTMUCH_CONFIG_MAIL_ROOT); if (prefix == NULL) goto FAIL; diff --git a/lib/message.cc b/lib/message.cc index 1bea90f0..35f0b87a 100644 --- a/lib/message.cc +++ b/lib/message.cc @@ -1098,7 +1098,7 @@ _notmuch_message_ensure_filename_list (notmuch_message_t *message) *colon = '\0'; - db_path = notmuch_database_get_path (message->notmuch); + db_path = notmuch_config_get (message->notmuch, NOTMUCH_CONFIG_MAIL_ROOT); directory = _notmuch_database_get_directory_path (local, message->notmuch, diff --git a/lib/notmuch.h b/lib/notmuch.h index 15ef77e7..3e0c0592 100644 --- a/lib/notmuch.h +++ b/lib/notmuch.h @@ -2473,6 +2473,7 @@ notmuch_config_list_destroy (notmuch_config_list_t *config_list); typedef enum _notmuch_config_key { NOTMUCH_CONFIG_FIRST, NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST, + NOTMUCH_CONFIG_MAIL_ROOT, NOTMUCH_CONFIG_HOOK_DIR, NOTMUCH_CONFIG_EXCLUDE_TAGS, NOTMUCH_CONFIG_NEW_TAGS, diff --git a/lib/open.cc b/lib/open.cc index bfeb1ce9..c881f45f 100644 --- a/lib/open.cc +++ b/lib/open.cc @@ -262,6 +262,11 @@ _choose_xapian_path (void *ctx, const char *database_path, const char **xapian_p if (status) goto DONE; + trial_path = talloc_asprintf (ctx, "%s/xapian", database_path); + status = _trial_open (trial_path, message_ptr); + if (status != NOTMUCH_STATUS_PATH_ERROR) + goto DONE; + notmuch_path = talloc_asprintf (ctx, "%s/.notmuch", database_path); status = _db_dir_exists (notmuch_path, message_ptr); if (status) diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh new file mode 100755 index 00000000..c6920ca9 --- /dev/null +++ b/test/T055-path-config.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +test_description='Configuration of mail-root and database path' +. $(dirname "$0")/test-lib.sh || exit 1 + +backup_config () { + local test_name=$(basename $0 .sh) + cp ${NOTMUCH_CONFIG} notmuch-config-backup.${test_name} +} + +restore_config () { + local test_name=$(basename $0 .sh) + export NOTMUCH_CONFIG="${TMP_DIRECTORY}/notmuch-config" + unset CONFIG_PATH + unset DATABASE_PATH + unset NOTMUCH_PROFILE + cp notmuch-config-backup.${test_name} ${NOTMUCH_CONFIG} +} + +split_config () { + local dir + backup_config + dir="$TMP_DIRECTORY/database.$test_count" + rm -rf $dir + mkdir $dir + notmuch config set database.path $dir + notmuch config set database.mail_root $MAIL_DIR + DATABASE_PATH=$dir +} + + + +for config in traditional split; do + # start each set of tests with a known set of messages + add_email_corpus + + case $config in + traditional) + backup_config + ;; + split) + split_config + mv mail/.notmuch/xapian $DATABASE_PATH + ;; + esac + + test_begin_subtest "count ($config)" + output=$(notmuch count '*') + test_expect_equal "$output" '52' + + test_begin_subtest "count+tag ($config)" + tag="tag${RANDOM}" + notmuch tag +$tag '*' + output=$(notmuch count tag:$tag) + notmuch tag -$tag '*' + test_expect_equal "$output" '52' + + test_begin_subtest "address ($config)" + notmuch address --deduplicate=no --sort=newest-first --output=sender --output=recipients path:foo >OUTPUT + cat <EXPECTED +Carl Worth +notmuch@notmuchmail.org +EOF + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "dump ($config)" + notmuch dump is:attachment and is:signed | sort > OUTPUT + cat < EXPECTED +#notmuch-dump batch-tag:3 config,properties,tags ++attachment +inbox +signed +unread -- id:20091118005829.GB25380@dottiness.seas.harvard.edu ++attachment +inbox +signed +unread -- id:20091118010116.GC25380@dottiness.seas.harvard.edu +EOF + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "dump + tag + restore ($config)" + notmuch dump '*' > EXPECTED + notmuch tag -inbox '*' + notmuch restore < EXPECTED + notmuch dump > OUTPUT + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "reindex ($config)" + notmuch search --output=messages '*' > EXPECTED + notmuch reindex '*' + notmuch search --output=messages '*' > OUTPUT + test_expect_equal_file EXPECTED OUTPUT + + restore_config +done + +test_done diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh index 4e510e97..5cf70987 100755 --- a/test/T590-libconfig.sh +++ b/test/T590-libconfig.sh @@ -364,6 +364,7 @@ EOF cat <<'EOF' >EXPECTED == stdout == MAIL_DIR +MAIL_DIR MAIL_DIR/.notmuch/hooks inbox;unread -- 2.30.1