[PATCH] man: add notes about issues with notmuch-count

[PATCH] man: add notes about issues with notmuch-count

[Sergei Shilovsky]

Here are some notes to 'notmuch-count' man page and a couple of related
suggestions/questions for possible discussion:

1. `notmuch count --output=(messages-exact|messages-estimate)`. Maybe with
notmuch_query_count_messages_exact function

2. extract `notmuch count --output=files` to  notmuch_query_count_filenames
library function

Sergei Shilovsky (1):
  man: add notes about issues with notmuch-count

 doc/man1/notmuch-count.rst | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

-- 
2.1.0

[PATCH] man: add notes about issues with notmuch-count

[Sergei Shilovsky]

1. notmuch count --output=messages outputs estimated number

2. notmuch count --output=(files|threads) can be slow
---
 doc/man1/notmuch-count.rst | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/doc/man1/notmuch-count.rst b/doc/man1/notmuch-count.rst
index ca78c18..3bb6981 100644
--- a/doc/man1/notmuch-count.rst
+++ b/doc/man1/notmuch-count.rst
@@ -10,7 +10,7 @@ SYNOPSIS
 DESCRIPTION
 ===========
 
-Count messages matching the search terms.
+Count (an estimate of) the number of messages matching the search terms.
 
 The number of matching messages (or threads) is output to stdout.
 
@@ -25,16 +25,20 @@ Supported options for **count** include
     ``--output=(messages|threads|files)``
 
         **messages**
-            Output the number of matching messages. This is the default.
+            Output an estimate of the number of matching messages. This is the
+            default.
 
         **threads**
-            Output the number of matching threads.
+            Output the exact number of matching threads. The search is
+            performed, so this is a significantly heavier operation than
+            ``--output=messages``.
 
         **files**
-            Output the number of files associated with matching
+            Output the exact number of files associated with matching
             messages. This may be bigger than the number of matching
             messages due to duplicates (i.e. multiple files having the
-            same message-id).
+            same message-id). The search is performed, so this is a
+            significantly heavier operation than ``--output=messages``.
 
     ``--exclude=(true|false)``
         Specify whether to omit messages matching search.tag\_exclude
-- 
2.1.0

[PATCH] lib: make notmuch_query_count_messages exact

[Jani Nikula]

Our tests have expected this to be exact all along, but maybe
inaccuracies only show up with big databases.
---
 devel/TODO    | 5 -----
 lib/notmuch.h | 6 +++---
 lib/query.cc  | 7 ++++++-
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/devel/TODO b/devel/TODO
index 1cf4089f1d93..116194d82ad1 100644
--- a/devel/TODO
+++ b/devel/TODO
@@ -188,11 +188,6 @@ into the shared-library interface.
 Audit all libnotmuch entry points to ensure that all Xapian calls are
 wrapped in a try/catch block.
 
-Fix the "count" functionality to be exact as Olly explained in IRC:
-
-	ojwb> cworth: if you set the check_at_least parameter to the
-	database size, get_matches_estimated() will be exact
-
 Fix the threading of a message that has a References: header but no
 In-Reply-To: header (see id:"87lixxnxpb.fsf@yoom.home.cworth.org").
 
diff --git a/lib/notmuch.h b/lib/notmuch.h
index dae041640fdb..6091082617e7 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -893,10 +893,10 @@ void
 notmuch_threads_destroy (notmuch_threads_t *threads);
 
 /**
- * Return an estimate of the number of messages matching a search.
+ * Return the number of messages matching a search.
  *
- * This function performs a search and returns Xapian's best
- * guess as to number of matching messages.
+ * This function performs a search and returns the number of matching
+ * messages.
  *
  * If a Xapian exception occurs, this function may return 0 (after
  * printing a message).
diff --git a/lib/query.cc b/lib/query.cc
index 60ff8bd9a39e..ce26e3f3e4ed 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -557,7 +557,12 @@ notmuch_query_count_messages (notmuch_query_t *query)
 
 	enquire.set_query (final_query);
 
-	mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount ());
+	/*
+	 * Set the checkatleast parameter to the number of documents
+	 * in the database to make get_matches_estimated() exact.
+	 */
+	mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount (),
+				 notmuch->xapian_db->get_doccount ());
 
 	count = mset.get_matches_estimated();
 
-- 
2.1.1

Re: [PATCH] lib: make notmuch_query_count_messages exact

[Jani Nikula]

On Tue, 14 Oct 2014, Jani Nikula <jani@nikula.org> wrote:
> Our tests have expected this to be exact all along, but maybe
> inaccuracies only show up with big databases.

From my IRC logs on Thu Oct 16 2014

j4ni   olly: does this do what it claims?
       http://mid.gmane.org/1413304374-17997-1-git-send-email-jani@nikula.org
olly   j4ni: not really
olly   if you don't set check at least it defaults to the number of
       requested matches
olly   so the code part is effectively a no-op
olly   but the reported number will be exact already so the comment
       changes are right
olly   i guess you could argue there's some merit in passing it explicitly
       in case someone decreases the request number of results

> ---
>  devel/TODO    | 5 -----
>  lib/notmuch.h | 6 +++---
>  lib/query.cc  | 7 ++++++-
>  3 files changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/devel/TODO b/devel/TODO
> index 1cf4089f1d93..116194d82ad1 100644
> --- a/devel/TODO
> +++ b/devel/TODO
> @@ -188,11 +188,6 @@ into the shared-library interface.
>  Audit all libnotmuch entry points to ensure that all Xapian calls are
>  wrapped in a try/catch block.
>  
> -Fix the "count" functionality to be exact as Olly explained in IRC:
> -
> -	ojwb> cworth: if you set the check_at_least parameter to the
> -	database size, get_matches_estimated() will be exact
> -
>  Fix the threading of a message that has a References: header but no
>  In-Reply-To: header (see id:"87lixxnxpb.fsf@yoom.home.cworth.org").
>  
> diff --git a/lib/notmuch.h b/lib/notmuch.h
> index dae041640fdb..6091082617e7 100644
> --- a/lib/notmuch.h
> +++ b/lib/notmuch.h
> @@ -893,10 +893,10 @@ void
>  notmuch_threads_destroy (notmuch_threads_t *threads);
>  
>  /**
> - * Return an estimate of the number of messages matching a search.
> + * Return the number of messages matching a search.
>   *
> - * This function performs a search and returns Xapian's best
> - * guess as to number of matching messages.
> + * This function performs a search and returns the number of matching
> + * messages.
>   *
>   * If a Xapian exception occurs, this function may return 0 (after
>   * printing a message).
> diff --git a/lib/query.cc b/lib/query.cc
> index 60ff8bd9a39e..ce26e3f3e4ed 100644
> --- a/lib/query.cc
> +++ b/lib/query.cc
> @@ -557,7 +557,12 @@ notmuch_query_count_messages (notmuch_query_t *query)
>  
>  	enquire.set_query (final_query);
>  
> -	mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount ());
> +	/*
> +	 * Set the checkatleast parameter to the number of documents
> +	 * in the database to make get_matches_estimated() exact.
> +	 */
> +	mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount (),
> +				 notmuch->xapian_db->get_doccount ());
>  
>  	count = mset.get_matches_estimated();
>  
> -- 
> 2.1.1

Re: [PATCH] lib: make notmuch_query_count_messages exact

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> Our tests have expected this to be exact all along, but maybe
> inaccuracies only show up with big databases.

pushed, with a completely rewritten commit message.

d