From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id AA9SJgW0Ll/JAwAA0tVLHw (envelope-from ) for ; Sat, 08 Aug 2020 14:17:41 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 6NofIgW0Ll/IaQAAbx9fmQ (envelope-from ) for ; Sat, 08 Aug 2020 14:17:41 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CEE539403D6 for ; Sat, 8 Aug 2020 14:17:40 +0000 (UTC) Received: from [144.217.243.247] (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 9983A29AED; Sat, 8 Aug 2020 10:17:17 -0400 (EDT) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 318762711D for ; Sat, 8 Aug 2020 10:17:10 -0400 (EDT) Received: by fethera.tethera.net (Postfix, from userid 1001) id 29D645FA66; Sat, 8 Aug 2020 10:17:10 -0400 (EDT) Received: (nullmailer pid 1124276 invoked by uid 1000); Sat, 08 Aug 2020 14:17:04 -0000 From: David Bremner To: notmuch@notmuchmail.org Cc: David Bremner Subject: [PATCH 10/19] lib: factor out prefix related code to its own file Date: Sat, 8 Aug 2020 11:16:44 -0300 Message-Id: <20200808141653.1124111-11-david@tethera.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200808141653.1124111-1-david@tethera.net> References: <20200808141653.1124111-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: KOFOPX557HBCPJPY7SRPVKR3CFR5QPHH X-Message-ID-Hash: KOFOPX557HBCPJPY7SRPVKR3CFR5QPHH X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Spam-Score: 0.03 X-TUID: 3fnvl75ItmmT Reduce the size of database.cc, and limit the scope of prefix_table, make sure it's accessed via a well-defined internal API. --- lib/Makefile.local | 4 +- lib/database-private.h | 7 ++ lib/database.cc | 204 ++------------------------------------- lib/prefix.cc | 210 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 229 insertions(+), 196 deletions(-) create mode 100644 lib/prefix.cc diff --git a/lib/Makefile.local b/lib/Makefile.local index b95ae216..15e71ef8 100644 --- a/lib/Makefile.local +++ b/lib/Makefile.local @@ -60,7 +60,9 @@ libnotmuch_cxx_srcs = \ $(dir)/regexp-fields.cc \ $(dir)/thread.cc \ $(dir)/thread-fp.cc \ - $(dir)/features.cc + $(dir)/features.cc \ + $(dir)/prefix.cc + libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o) diff --git a/lib/database-private.h b/lib/database-private.h index 3b365c41..d83cf0d0 100644 --- a/lib/database-private.h +++ b/lib/database-private.h @@ -280,4 +280,11 @@ _notmuch_database_parse_features (const void *ctx, const char *features, unsigne char * _notmuch_database_print_features (const void *ctx, unsigned int features); +/* prefix.cc */ +notmuch_status_t +_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch); + +notmuch_status_t +_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch); + #endif diff --git a/lib/database.cc b/lib/database.cc index ceba6dd5..0620de5a 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -263,80 +263,6 @@ _notmuch_database_mode (notmuch_database_t *notmuch) * same thread. */ -/* With these prefix values we follow the conventions published here: - * - * https://xapian.org/docs/omega/termprefixes.html - * - * as much as makes sense. Note that I took some liberty in matching - * the reserved prefix values to notmuch concepts, (for example, 'G' - * is documented as "newsGroup (or similar entity - e.g. a web forum - * name)", for which I think the thread is the closest analogue in - * notmuch. This in spite of the fact that we will eventually be - * storing mailing-list messages where 'G' for "mailing list name" - * might be even a closer analogue. I'm treating the single-character - * prefixes preferentially for core notmuch concepts (which will be - * nearly universal to all mail messages). - */ - -static const -prefix_t prefix_table[] = { - /* name term prefix flags */ - { "type", "T", NOTMUCH_FIELD_NO_FLAGS }, - { "reference", "XREFERENCE", NOTMUCH_FIELD_NO_FLAGS }, - { "replyto", "XREPLYTO", NOTMUCH_FIELD_NO_FLAGS }, - { "directory", "XDIRECTORY", NOTMUCH_FIELD_NO_FLAGS }, - { "file-direntry", "XFDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, - { "directory-direntry", "XDDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, - { "body", "", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "thread", "G", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "tag", "K", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "is", "K", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "id", "Q", NOTMUCH_FIELD_EXTERNAL }, - { "mid", "Q", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "path", "P", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "property", "XPROPERTY", NOTMUCH_FIELD_EXTERNAL }, - /* - * Unconditionally add ':' to reduce potential ambiguity with - * overlapping prefixes and/or terms that start with capital - * letters. See Xapian document termprefixes.html for related - * discussion. - */ - { "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "date", NULL, NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "query", NULL, NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "from", "XFROM", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC | - NOTMUCH_FIELD_PROCESSOR }, - { "to", "XTO", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "attachment", "XATTACHMENT", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "mimetype", "XMIMETYPE", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "subject", "XSUBJECT", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC | - NOTMUCH_FIELD_PROCESSOR }, -}; - -static void -_setup_query_field_default (const prefix_t *prefix, notmuch_database_t *notmuch) -{ - if (prefix->prefix) - notmuch->query_parser->add_prefix ("", prefix->prefix); - if (prefix->flags & NOTMUCH_FIELD_PROBABILISTIC) - notmuch->query_parser->add_prefix (prefix->name, prefix->prefix); - else - notmuch->query_parser->add_boolean_prefix (prefix->name, prefix->prefix); -} notmuch_string_map_iterator_t * _notmuch_database_user_headers (notmuch_database_t *notmuch) @@ -344,118 +270,6 @@ _notmuch_database_user_headers (notmuch_database_t *notmuch) return _notmuch_string_map_iterator_create (notmuch->user_header, "", false); } -const char * -_user_prefix (void *ctx, const char *name) -{ - return talloc_asprintf (ctx, "XU%s:", name); -} - -static notmuch_status_t -_setup_user_query_fields (notmuch_database_t *notmuch) -{ - notmuch_config_list_t *list; - notmuch_status_t status; - - notmuch->user_prefix = _notmuch_string_map_create (notmuch); - if (notmuch->user_prefix == NULL) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - - notmuch->user_header = _notmuch_string_map_create (notmuch); - if (notmuch->user_header == NULL) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - - status = notmuch_database_get_config_list (notmuch, CONFIG_HEADER_PREFIX, &list); - if (status) - return status; - - for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { - - prefix_t query_field; - - const char *key = notmuch_config_list_key (list) - + sizeof (CONFIG_HEADER_PREFIX) - 1; - - _notmuch_string_map_append (notmuch->user_prefix, - key, - _user_prefix (notmuch, key)); - - _notmuch_string_map_append (notmuch->user_header, - key, - notmuch_config_list_value (list)); - - query_field.name = talloc_strdup (notmuch, key); - query_field.prefix = _user_prefix (notmuch, key); - query_field.flags = NOTMUCH_FIELD_PROBABILISTIC - | NOTMUCH_FIELD_EXTERNAL; - - _setup_query_field_default (&query_field, notmuch); - } - - notmuch_config_list_destroy (list); - - return NOTMUCH_STATUS_SUCCESS; -} - -static void -_setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) -{ - if (prefix->flags & NOTMUCH_FIELD_PROCESSOR) { - Xapian::FieldProcessor *fp; - - if (STRNCMP_LITERAL (prefix->name, "date") == 0) - fp = (new DateFieldProcessor(NOTMUCH_VALUE_TIMESTAMP))->release (); - else if (STRNCMP_LITERAL(prefix->name, "query") == 0) - fp = (new QueryFieldProcessor (*notmuch->query_parser, notmuch))->release (); - else if (STRNCMP_LITERAL (prefix->name, "thread") == 0) - fp = (new ThreadFieldProcessor (*notmuch->query_parser, notmuch))->release (); - else - fp = (new RegexpFieldProcessor (prefix->name, prefix->flags, - *notmuch->query_parser, notmuch))->release (); - - /* we treat all field-processor fields as boolean in order to get the raw input */ - if (prefix->prefix) - notmuch->query_parser->add_prefix ("", prefix->prefix); - notmuch->query_parser->add_boolean_prefix (prefix->name, fp); - } else { - _setup_query_field_default (prefix, notmuch); - } -} - -const char * -_find_prefix (const char *name) -{ - unsigned int i; - - for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { - if (strcmp (name, prefix_table[i].name) == 0) - return prefix_table[i].prefix; - } - - INTERNAL_ERROR ("No prefix exists for '%s'\n", name); - - return ""; -} - -/* Like find prefix, but include the possibility of user defined - * prefixes specific to this database */ - -const char * -_notmuch_database_prefix (notmuch_database_t *notmuch, const char *name) -{ - unsigned int i; - - /*XXX TODO: reduce code duplication */ - for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { - if (strcmp (name, prefix_table[i].name) == 0) - return prefix_table[i].prefix; - } - - if (notmuch->user_prefix) - return _notmuch_string_map_get (notmuch->user_prefix, name); - - return NULL; -} - const char * notmuch_status_to_string (notmuch_status_t status) { @@ -825,7 +639,7 @@ notmuch_database_open_with_config (const char *database_path, char *message = NULL; struct stat st; int err; - unsigned int i, version; + unsigned version; GKeyFile *key_file = NULL; static int initialized = 0; @@ -926,7 +740,7 @@ notmuch_database_open_with_config (const char *database_path, /* Check features. */ incompat_features = NULL; - notmuch->features = _parse_features ( + notmuch->features = _notmuch_database_parse_features ( local, notmuch->xapian_db->get_metadata ("features").c_str (), version, mode == NOTMUCH_DATABASE_MODE_READ_WRITE ? 'w' : 'r', &incompat_features); @@ -980,13 +794,13 @@ notmuch_database_open_with_config (const char *database_path, notmuch->query_parser->add_rangeprocessor (notmuch->date_range_processor); notmuch->query_parser->add_rangeprocessor (notmuch->last_mod_range_processor); - for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { - const prefix_t *prefix = &prefix_table[i]; - if (prefix->flags & NOTMUCH_FIELD_EXTERNAL) { - _setup_query_field (prefix, notmuch); - } - } - status = _setup_user_query_fields (notmuch); + status = _notmuch_database_setup_standard_query_fields (notmuch); + if (status) + goto DONE; + + status = _notmuch_database_setup_user_query_fields (notmuch); + if (status) + goto DONE; if (status == NOTMUCH_STATUS_SUCCESS) status = _notmuch_config_load_from_database (notmuch); diff --git a/lib/prefix.cc b/lib/prefix.cc new file mode 100644 index 00000000..dd7b193d --- /dev/null +++ b/lib/prefix.cc @@ -0,0 +1,210 @@ +#include "database-private.h" +#include "query-fp.h" +#include "thread-fp.h" +#include "regexp-fields.h" +#include "parse-time-vrp.h" + +typedef struct { + const char *name; + const char *prefix; + notmuch_field_flag_t flags; +} prefix_t; + +/* With these prefix values we follow the conventions published here: + * + * https://xapian.org/docs/omega/termprefixes.html + * + * as much as makes sense. Note that I took some liberty in matching + * the reserved prefix values to notmuch concepts, (for example, 'G' + * is documented as "newsGroup (or similar entity - e.g. a web forum + * name)", for which I think the thread is the closest analogue in + * notmuch. This in spite of the fact that we will eventually be + * storing mailing-list messages where 'G' for "mailing list name" + * might be even a closer analogue. I'm treating the single-character + * prefixes preferentially for core notmuch concepts (which will be + * nearly universal to all mail messages). + */ + +static const +prefix_t prefix_table[] = { + /* name term prefix flags */ + { "type", "T", NOTMUCH_FIELD_NO_FLAGS }, + { "reference", "XREFERENCE", NOTMUCH_FIELD_NO_FLAGS }, + { "replyto", "XREPLYTO", NOTMUCH_FIELD_NO_FLAGS }, + { "directory", "XDIRECTORY", NOTMUCH_FIELD_NO_FLAGS }, + { "file-direntry", "XFDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, + { "directory-direntry", "XDDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, + { "body", "", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "thread", "G", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "tag", "K", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "is", "K", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "id", "Q", NOTMUCH_FIELD_EXTERNAL }, + { "mid", "Q", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "path", "P", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "property", "XPROPERTY", NOTMUCH_FIELD_EXTERNAL }, + /* + * Unconditionally add ':' to reduce potential ambiguity with + * overlapping prefixes and/or terms that start with capital + * letters. See Xapian document termprefixes.html for related + * discussion. + */ + { "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "date", NULL, NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "query", NULL, NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "from", "XFROM", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC | + NOTMUCH_FIELD_PROCESSOR }, + { "to", "XTO", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "attachment", "XATTACHMENT", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "mimetype", "XMIMETYPE", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "subject", "XSUBJECT", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC | + NOTMUCH_FIELD_PROCESSOR }, +}; + +static const char * +_user_prefix (void *ctx, const char *name) +{ + return talloc_asprintf (ctx, "XU%s:", name); +} + +const char * +_find_prefix (const char *name) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { + if (strcmp (name, prefix_table[i].name) == 0) + return prefix_table[i].prefix; + } + + INTERNAL_ERROR ("No prefix exists for '%s'\n", name); + + return ""; +} + +/* Like find prefix, but include the possibility of user defined + * prefixes specific to this database */ + +const char * +_notmuch_database_prefix (notmuch_database_t *notmuch, const char *name) +{ + unsigned int i; + + /*XXX TODO: reduce code duplication */ + for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { + if (strcmp (name, prefix_table[i].name) == 0) + return prefix_table[i].prefix; + } + + if (notmuch->user_prefix) + return _notmuch_string_map_get (notmuch->user_prefix, name); + + return NULL; +} + +static void +_setup_query_field_default (const prefix_t *prefix, notmuch_database_t *notmuch) +{ + if (prefix->prefix) + notmuch->query_parser->add_prefix ("", prefix->prefix); + if (prefix->flags & NOTMUCH_FIELD_PROBABILISTIC) + notmuch->query_parser->add_prefix (prefix->name, prefix->prefix); + else + notmuch->query_parser->add_boolean_prefix (prefix->name, prefix->prefix); +} + +static void +_setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) +{ + if (prefix->flags & NOTMUCH_FIELD_PROCESSOR) { + Xapian::FieldProcessor *fp; + + if (STRNCMP_LITERAL (prefix->name, "date") == 0) + fp = (new DateFieldProcessor(NOTMUCH_VALUE_TIMESTAMP))->release (); + else if (STRNCMP_LITERAL(prefix->name, "query") == 0) + fp = (new QueryFieldProcessor (*notmuch->query_parser, notmuch))->release (); + else if (STRNCMP_LITERAL (prefix->name, "thread") == 0) + fp = (new ThreadFieldProcessor (*notmuch->query_parser, notmuch))->release (); + else + fp = (new RegexpFieldProcessor (prefix->name, prefix->flags, + *notmuch->query_parser, notmuch))->release (); + + /* we treat all field-processor fields as boolean in order to get the raw input */ + if (prefix->prefix) + notmuch->query_parser->add_prefix ("", prefix->prefix); + notmuch->query_parser->add_boolean_prefix (prefix->name, fp); + } else { + _setup_query_field_default (prefix, notmuch); + } +} + +notmuch_status_t +_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch) +{ + for (unsigned int i = 0; i < ARRAY_SIZE (prefix_table); i++) { + const prefix_t *prefix = &prefix_table[i]; + if (prefix->flags & NOTMUCH_FIELD_EXTERNAL) { + _setup_query_field (prefix, notmuch); + } + } + return NOTMUCH_STATUS_SUCCESS; +} + +notmuch_status_t +_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch) +{ + notmuch_config_list_t *list; + notmuch_status_t status; + + notmuch->user_prefix = _notmuch_string_map_create (notmuch); + if (notmuch->user_prefix == NULL) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + notmuch->user_header = _notmuch_string_map_create (notmuch); + if (notmuch->user_header == NULL) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + status = notmuch_database_get_config_list (notmuch, CONFIG_HEADER_PREFIX, &list); + if (status) + return status; + + for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { + + prefix_t query_field; + + const char *key = notmuch_config_list_key (list) + + sizeof (CONFIG_HEADER_PREFIX) - 1; + + _notmuch_string_map_append (notmuch->user_prefix, + key, + _user_prefix (notmuch, key)); + + _notmuch_string_map_append (notmuch->user_header, + key, + notmuch_config_list_value (list)); + + query_field.name = talloc_strdup (notmuch, key); + query_field.prefix = _user_prefix (notmuch, key); + query_field.flags = NOTMUCH_FIELD_PROBABILISTIC + | NOTMUCH_FIELD_EXTERNAL; + + _setup_query_field_default (&query_field, notmuch); + } + + notmuch_config_list_destroy (list); + + return NOTMUCH_STATUS_SUCCESS; +} -- 2.28.0