* [PATCH] notmuch compact support @ 2012-10-17 15:28 Ben Gamari 2012-10-17 15:28 ` [PATCH 1/3] Add notmuch_database_close_compact Ben Gamari ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Ben Gamari @ 2012-10-17 15:28 UTC (permalink / raw) To: notmuch Here is a new spin of my patchset introduced in late August (id:"1345476704-17091-1-git-send-email-bgamari.foss@gmail.com") adding a compact command to libnotmuch and the command line frontend. I believe the concerns raised in August have been addressed, but correct me if I'm wrong. Cheers, - Ben ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/3] Add notmuch_database_close_compact 2012-10-17 15:28 [PATCH] notmuch compact support Ben Gamari @ 2012-10-17 15:28 ` Ben Gamari 2012-10-17 18:56 ` Jani Nikula 2012-10-18 6:44 ` Tomi Ollila 2012-10-17 15:28 ` [PATCH 2/3] Produce status messages during compacting Ben Gamari 2012-10-17 15:28 ` [PATCH 3/3] Add notmuch compact command Ben Gamari 2 siblings, 2 replies; 10+ messages in thread From: Ben Gamari @ 2012-10-17 15:28 UTC (permalink / raw) To: notmuch --- configure | 21 ++++++++++++++++++++- lib/database.cc | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/notmuch.h | 14 ++++++++++++++ 3 files changed, 88 insertions(+), 1 deletion(-) diff --git a/configure b/configure index acb90a8..6551b13 100755 --- a/configure +++ b/configure @@ -270,7 +270,8 @@ printf "Checking for Xapian development files... " have_xapian=0 for xapian_config in ${XAPIAN_CONFIG}; do if ${xapian_config} --version > /dev/null 2>&1; then - printf "Yes (%s).\n" $(${xapian_config} --version | sed -e 's/.* //') + xapian_version=$(${xapian_config} --version | sed -e 's/.* //') + printf "Yes (%s).\n" ${xapian_version} have_xapian=1 xapian_cxxflags=$(${xapian_config} --cxxflags) xapian_ldflags=$(${xapian_config} --libs) @@ -282,6 +283,20 @@ if [ ${have_xapian} = "0" ]; then errors=$((errors + 1)) fi +have_xapian_compact=0 +if [ ${have_xapian} = "1" ]; then + printf "Checking for Xapian compact support... " + case "${xapian_version}" in + 0.*|1.[01].*|1.2.[0-5]) + printf "No.\n" ;; + [1-9]*.[0-9]*.[0-9]*) + have_xapian_compact=1 + printf "Yes.\n" ;; + *) + printf "Unknown version.\n" ;; + esac +fi + printf "Checking for GMime development files... " have_gmime=0 IFS=';' @@ -679,6 +694,9 @@ LINKER_RESOLVES_LIBRARY_DEPENDENCIES = ${linker_resolves_library_dependencies} XAPIAN_CXXFLAGS = ${xapian_cxxflags} XAPIAN_LDFLAGS = ${xapian_ldflags} +# Whether compact is supported by this version of Xapian +HAVE_XAPIAN_COMPACT = ${have_xapian_compact} + # Flags needed to compile and link against GMime-2.4 GMIME_CFLAGS = ${gmime_cflags} GMIME_LDFLAGS = ${gmime_ldflags} @@ -715,6 +733,7 @@ CONFIGURE_CFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ \$(TALLOC_CFLAGS) -DHAVE_VALGRIND=\$(HAVE_VALGRIND) \\ \$(VALGRIND_CFLAGS) \$(XAPIAN_CXXFLAGS) \\ + -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT) \\ -DHAVE_STRCASESTR=\$(HAVE_STRCASESTR) CONFIGURE_LDFLAGS = \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(XAPIAN_LDFLAGS) EOF diff --git a/lib/database.cc b/lib/database.cc index 761dc1a..6e83a61 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -781,6 +781,60 @@ notmuch_database_close (notmuch_database_t *notmuch) } void +notmuch_database_close_compact (notmuch_database_t *notmuch) +{ + void *local = talloc_new (NULL); + Xapian::Compactor compactor; + char *notmuch_path, *xapian_path, *compact_xapian_path, *old_xapian_path; + +#if HAVE_XAPIAN_COMPACT + if (! (notmuch_path = talloc_asprintf (local, "%s/%s", notmuch->path, ".notmuch"))) { + fprintf (stderr, "Out of memory\n"); + goto DONE; + } + + if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, "xapian"))) { + fprintf (stderr, "Out of memory\n"); + goto DONE; + } + + if (! (compact_xapian_path = talloc_asprintf (local, "%s.compact", xapian_path))) { + fprintf (stderr, "Out of memory\n"); + goto DONE; + } + + if (! (old_xapian_path = talloc_asprintf (local, "%s.old", xapian_path))) { + fprintf (stderr, "Out of memory\n"); + goto DONE; + } + + try { + compactor.set_renumber(false); + compactor.add_source(xapian_path); + compactor.set_destdir(compact_xapian_path); + compactor.compact(); + + if (rename(xapian_path, old_xapian_path)) { + fprintf (stderr, "Error moving old database out of the way\n"); + goto DONE; + } + + if (rename(compact_xapian_path, xapian_path)) { + fprintf (stderr, "Error moving compacted database\n"); + goto DONE; + } + } catch (Xapian::InvalidArgumentError e) { + fprintf (stderr, "Error while compacting: %s", e.get_msg().c_str()); + } + +#endif + + notmuch_database_close(notmuch); +DONE: + talloc_free(local); +} + +void notmuch_database_destroy (notmuch_database_t *notmuch) { notmuch_database_close (notmuch); diff --git a/lib/notmuch.h b/lib/notmuch.h index 3633bed..50babfb 100644 --- a/lib/notmuch.h +++ b/lib/notmuch.h @@ -215,6 +215,20 @@ notmuch_database_open (const char *path, void notmuch_database_close (notmuch_database_t *database); +/* Close the given notmuch database and then compact it. + * + * After notmuch_database_close_compact has been called, calls to + * other functions on objects derived from this database may either + * behave as if the database had not been closed (e.g., if the + * required data has been cached) or may fail with a + * NOTMUCH_STATUS_XAPIAN_EXCEPTION. + * + * notmuch_database_close_compact can be called multiple times. Later + * calls have no effect. + */ +void +notmuch_database_close_compact (notmuch_database_t *notmuch); + /* Destroy the notmuch database, closing it if necessary and freeing * all associated resources. */ void -- 1.7.10.4 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 1/3] Add notmuch_database_close_compact 2012-10-17 15:28 ` [PATCH 1/3] Add notmuch_database_close_compact Ben Gamari @ 2012-10-17 18:56 ` Jani Nikula 2012-10-18 6:44 ` Tomi Ollila 1 sibling, 0 replies; 10+ messages in thread From: Jani Nikula @ 2012-10-17 18:56 UTC (permalink / raw) To: Ben Gamari, notmuch Hi Ben - I'd like some meaningful commit messages here. Please find other comments inline. BR, Jani. On Wed, 17 Oct 2012, Ben Gamari <bgamari.foss@gmail.com> wrote: > --- > configure | 21 ++++++++++++++++++++- > lib/database.cc | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ > lib/notmuch.h | 14 ++++++++++++++ > 3 files changed, 88 insertions(+), 1 deletion(-) > > diff --git a/configure b/configure > index acb90a8..6551b13 100755 > --- a/configure > +++ b/configure > @@ -270,7 +270,8 @@ printf "Checking for Xapian development files... " > have_xapian=0 > for xapian_config in ${XAPIAN_CONFIG}; do > if ${xapian_config} --version > /dev/null 2>&1; then > - printf "Yes (%s).\n" $(${xapian_config} --version | sed -e 's/.* //') > + xapian_version=$(${xapian_config} --version | sed -e 's/.* //') > + printf "Yes (%s).\n" ${xapian_version} > have_xapian=1 > xapian_cxxflags=$(${xapian_config} --cxxflags) > xapian_ldflags=$(${xapian_config} --libs) > @@ -282,6 +283,20 @@ if [ ${have_xapian} = "0" ]; then > errors=$((errors + 1)) > fi > > +have_xapian_compact=0 > +if [ ${have_xapian} = "1" ]; then > + printf "Checking for Xapian compact support... " > + case "${xapian_version}" in > + 0.*|1.[01].*|1.2.[0-5]) > + printf "No.\n" ;; > + [1-9]*.[0-9]*.[0-9]*) > + have_xapian_compact=1 > + printf "Yes.\n" ;; > + *) > + printf "Unknown version.\n" ;; > + esac > +fi > + > printf "Checking for GMime development files... " > have_gmime=0 > IFS=';' > @@ -679,6 +694,9 @@ LINKER_RESOLVES_LIBRARY_DEPENDENCIES = ${linker_resolves_library_dependencies} > XAPIAN_CXXFLAGS = ${xapian_cxxflags} > XAPIAN_LDFLAGS = ${xapian_ldflags} > > +# Whether compact is supported by this version of Xapian > +HAVE_XAPIAN_COMPACT = ${have_xapian_compact} > + > # Flags needed to compile and link against GMime-2.4 > GMIME_CFLAGS = ${gmime_cflags} > GMIME_LDFLAGS = ${gmime_ldflags} > @@ -715,6 +733,7 @@ CONFIGURE_CFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ > CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ > \$(TALLOC_CFLAGS) -DHAVE_VALGRIND=\$(HAVE_VALGRIND) \\ > \$(VALGRIND_CFLAGS) \$(XAPIAN_CXXFLAGS) \\ > + -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT) \\ > -DHAVE_STRCASESTR=\$(HAVE_STRCASESTR) > CONFIGURE_LDFLAGS = \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(XAPIAN_LDFLAGS) > EOF > diff --git a/lib/database.cc b/lib/database.cc > index 761dc1a..6e83a61 100644 > --- a/lib/database.cc > +++ b/lib/database.cc > @@ -781,6 +781,60 @@ notmuch_database_close (notmuch_database_t *notmuch) > } > > void > +notmuch_database_close_compact (notmuch_database_t *notmuch) It is clear that there are error conditions, so this should be of type notmuch_status_t. Even if you don't know what to do with the errors now, you'll find that changing the API is a PITA later. > +{ > + void *local = talloc_new (NULL); > + Xapian::Compactor compactor; > + char *notmuch_path, *xapian_path, *compact_xapian_path, *old_xapian_path; > + > +#if HAVE_XAPIAN_COMPACT > + if (! (notmuch_path = talloc_asprintf (local, "%s/%s", notmuch->path, ".notmuch"))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } You could include notmuch->path and .notmuch above into the asprintf below. A personal preference, I think this style would be much more readable: xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, "xapian"); if (!xapian_path) { ... } > + > + if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, "xapian"))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + if (! (compact_xapian_path = talloc_asprintf (local, "%s.compact", xapian_path))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + if (! (old_xapian_path = talloc_asprintf (local, "%s.old", xapian_path))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + try { > + compactor.set_renumber(false); > + compactor.add_source(xapian_path); > + compactor.set_destdir(compact_xapian_path); > + compactor.compact(); > + > + if (rename(xapian_path, old_xapian_path)) { > + fprintf (stderr, "Error moving old database out of the way\n"); > + goto DONE; > + } > + > + if (rename(compact_xapian_path, xapian_path)) { > + fprintf (stderr, "Error moving compacted database\n"); > + goto DONE; > + } Please add strerror(errno) into the two error prints above. The user would probably like to know why the errors occurred. > + } catch (Xapian::InvalidArgumentError e) { Should this be catch (const Xapian::Error &error) ? > + fprintf (stderr, "Error while compacting: %s", e.get_msg().c_str()); > + } > + > +#endif > + > + notmuch_database_close(notmuch); The database gets closed on Xapian errors, but not on talloc or rename errors, and the caller has no way of knowing. See the return status above, but also read on... > +DONE: > + talloc_free(local); > +} > + > +void > notmuch_database_destroy (notmuch_database_t *notmuch) > { > notmuch_database_close (notmuch); > diff --git a/lib/notmuch.h b/lib/notmuch.h > index 3633bed..50babfb 100644 > --- a/lib/notmuch.h > +++ b/lib/notmuch.h > @@ -215,6 +215,20 @@ notmuch_database_open (const char *path, > void > notmuch_database_close (notmuch_database_t *database); > > +/* Close the given notmuch database and then compact it. It's the other way round, isn't it? But do we want the call to do two things, one of which we already have a call for (closing the database)? That's not orthogonal. Maybe closing the database after compaction is the right thing to do (is it?) but even so, could we leave that responsibility to the API user? The caller does have to open the database too, and calls to open, compact, close seem good. > + * > + * After notmuch_database_close_compact has been called, calls to > + * other functions on objects derived from this database may either > + * behave as if the database had not been closed (e.g., if the > + * required data has been cached) or may fail with a > + * NOTMUCH_STATUS_XAPIAN_EXCEPTION. > + * > + * notmuch_database_close_compact can be called multiple times. Later > + * calls have no effect. > + */ > +void > +notmuch_database_close_compact (notmuch_database_t *notmuch); > + > /* Destroy the notmuch database, closing it if necessary and freeing > * all associated resources. */ > void > -- > 1.7.10.4 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/3] Add notmuch_database_close_compact 2012-10-17 15:28 ` [PATCH 1/3] Add notmuch_database_close_compact Ben Gamari 2012-10-17 18:56 ` Jani Nikula @ 2012-10-18 6:44 ` Tomi Ollila 1 sibling, 0 replies; 10+ messages in thread From: Tomi Ollila @ 2012-10-18 6:44 UTC (permalink / raw) To: Ben Gamari, notmuch On Wed, Oct 17 2012, Ben Gamari <bgamari.foss@gmail.com> wrote: > --- > configure | 21 ++++++++++++++++++++- > lib/database.cc | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ > lib/notmuch.h | 14 ++++++++++++++ > 3 files changed, 88 insertions(+), 1 deletion(-) > > diff --git a/configure b/configure > index acb90a8..6551b13 100755 > --- a/configure > +++ b/configure > @@ -270,7 +270,8 @@ printf "Checking for Xapian development files... " > have_xapian=0 > for xapian_config in ${XAPIAN_CONFIG}; do > if ${xapian_config} --version > /dev/null 2>&1; then > - printf "Yes (%s).\n" $(${xapian_config} --version | sed -e 's/.* //') > + xapian_version=$(${xapian_config} --version | sed -e 's/.* //') > + printf "Yes (%s).\n" ${xapian_version} > have_xapian=1 > xapian_cxxflags=$(${xapian_config} --cxxflags) > xapian_ldflags=$(${xapian_config} --libs) > @@ -282,6 +283,20 @@ if [ ${have_xapian} = "0" ]; then > errors=$((errors + 1)) > fi > > +have_xapian_compact=0 > +if [ ${have_xapian} = "1" ]; then > + printf "Checking for Xapian compact support... " > + case "${xapian_version}" in > + 0.*|1.[01].*|1.2.[0-5]) > + printf "No.\n" ;; > + [1-9]*.[0-9]*.[0-9]*) > + have_xapian_compact=1 > + printf "Yes.\n" ;; > + *) > + printf "Unknown version.\n" ;; > + esac > +fi This is pretty good approximation(*) for the version check, but some comments are in place, like mentioning that compaction is supported in Xapian versions 1.2.6 and newer (as the code is not obvious to casual reader). (*) The match for yes part gives some room for non-digit version parts (if that would ever be needed) and also it rules out (most) cases where garbage were assigned to xapian_version. Tomi > + > printf "Checking for GMime development files... " > have_gmime=0 > IFS=';' > @@ -679,6 +694,9 @@ LINKER_RESOLVES_LIBRARY_DEPENDENCIES = ${linker_resolves_library_dependencies} > XAPIAN_CXXFLAGS = ${xapian_cxxflags} > XAPIAN_LDFLAGS = ${xapian_ldflags} > > +# Whether compact is supported by this version of Xapian > +HAVE_XAPIAN_COMPACT = ${have_xapian_compact} > + > # Flags needed to compile and link against GMime-2.4 > GMIME_CFLAGS = ${gmime_cflags} > GMIME_LDFLAGS = ${gmime_ldflags} > @@ -715,6 +733,7 @@ CONFIGURE_CFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ > CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ > \$(TALLOC_CFLAGS) -DHAVE_VALGRIND=\$(HAVE_VALGRIND) \\ > \$(VALGRIND_CFLAGS) \$(XAPIAN_CXXFLAGS) \\ > + -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT) \\ > -DHAVE_STRCASESTR=\$(HAVE_STRCASESTR) > CONFIGURE_LDFLAGS = \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(XAPIAN_LDFLAGS) > EOF > diff --git a/lib/database.cc b/lib/database.cc > index 761dc1a..6e83a61 100644 > --- a/lib/database.cc > +++ b/lib/database.cc > @@ -781,6 +781,60 @@ notmuch_database_close (notmuch_database_t *notmuch) > } > > void > +notmuch_database_close_compact (notmuch_database_t *notmuch) > +{ > + void *local = talloc_new (NULL); > + Xapian::Compactor compactor; > + char *notmuch_path, *xapian_path, *compact_xapian_path, *old_xapian_path; > + > +#if HAVE_XAPIAN_COMPACT > + if (! (notmuch_path = talloc_asprintf (local, "%s/%s", notmuch->path, ".notmuch"))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, "xapian"))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + if (! (compact_xapian_path = talloc_asprintf (local, "%s.compact", xapian_path))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + if (! (old_xapian_path = talloc_asprintf (local, "%s.old", xapian_path))) { > + fprintf (stderr, "Out of memory\n"); > + goto DONE; > + } > + > + try { > + compactor.set_renumber(false); > + compactor.add_source(xapian_path); > + compactor.set_destdir(compact_xapian_path); > + compactor.compact(); > + > + if (rename(xapian_path, old_xapian_path)) { > + fprintf (stderr, "Error moving old database out of the way\n"); > + goto DONE; > + } > + > + if (rename(compact_xapian_path, xapian_path)) { > + fprintf (stderr, "Error moving compacted database\n"); > + goto DONE; > + } > + } catch (Xapian::InvalidArgumentError e) { > + fprintf (stderr, "Error while compacting: %s", e.get_msg().c_str()); > + } > + > +#endif > + > + notmuch_database_close(notmuch); > +DONE: > + talloc_free(local); > +} > + > +void > notmuch_database_destroy (notmuch_database_t *notmuch) > { > notmuch_database_close (notmuch); > diff --git a/lib/notmuch.h b/lib/notmuch.h > index 3633bed..50babfb 100644 > --- a/lib/notmuch.h > +++ b/lib/notmuch.h > @@ -215,6 +215,20 @@ notmuch_database_open (const char *path, > void > notmuch_database_close (notmuch_database_t *database); > > +/* Close the given notmuch database and then compact it. > + * > + * After notmuch_database_close_compact has been called, calls to > + * other functions on objects derived from this database may either > + * behave as if the database had not been closed (e.g., if the > + * required data has been cached) or may fail with a > + * NOTMUCH_STATUS_XAPIAN_EXCEPTION. > + * > + * notmuch_database_close_compact can be called multiple times. Later > + * calls have no effect. > + */ > +void > +notmuch_database_close_compact (notmuch_database_t *notmuch); > + > /* Destroy the notmuch database, closing it if necessary and freeing > * all associated resources. */ > void > -- > 1.7.10.4 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 2/3] Produce status messages during compacting 2012-10-17 15:28 [PATCH] notmuch compact support Ben Gamari 2012-10-17 15:28 ` [PATCH 1/3] Add notmuch_database_close_compact Ben Gamari @ 2012-10-17 15:28 ` Ben Gamari 2012-10-17 18:59 ` Jani Nikula 2012-10-17 15:28 ` [PATCH 3/3] Add notmuch compact command Ben Gamari 2 siblings, 1 reply; 10+ messages in thread From: Ben Gamari @ 2012-10-17 15:28 UTC (permalink / raw) To: notmuch --- lib/database.cc | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/lib/database.cc b/lib/database.cc index 6e83a61..49aa36d 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -780,11 +780,24 @@ notmuch_database_close (notmuch_database_t *notmuch) notmuch->value_range_processor = NULL; } +class NotmuchCompactor : public Xapian::Compactor +{ +public: + virtual void + set_status (const std::string &table, const std::string &status) + { + if (status.length() == 0) + fprintf (stderr, "compacting table %s:\n", table.c_str()); + else + fprintf (stderr, " %s\n", status.c_str()); + } +}; + void notmuch_database_close_compact (notmuch_database_t *notmuch) { void *local = talloc_new (NULL); - Xapian::Compactor compactor; + NotmuchCompactor compactor; char *notmuch_path, *xapian_path, *compact_xapian_path, *old_xapian_path; #if HAVE_XAPIAN_COMPACT -- 1.7.10.4 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] Produce status messages during compacting 2012-10-17 15:28 ` [PATCH 2/3] Produce status messages during compacting Ben Gamari @ 2012-10-17 18:59 ` Jani Nikula 0 siblings, 0 replies; 10+ messages in thread From: Jani Nikula @ 2012-10-17 18:59 UTC (permalink / raw) To: Ben Gamari, notmuch Again, a commit message saying *why* having a flood of status messages is a good idea would be appreciated. I'm not sure it is a good idea. On Wed, 17 Oct 2012, Ben Gamari <bgamari.foss@gmail.com> wrote: > --- > lib/database.cc | 15 ++++++++++++++- > 1 file changed, 14 insertions(+), 1 deletion(-) > > diff --git a/lib/database.cc b/lib/database.cc > index 6e83a61..49aa36d 100644 > --- a/lib/database.cc > +++ b/lib/database.cc > @@ -780,11 +780,24 @@ notmuch_database_close (notmuch_database_t *notmuch) > notmuch->value_range_processor = NULL; > } > > +class NotmuchCompactor : public Xapian::Compactor > +{ > +public: > + virtual void > + set_status (const std::string &table, const std::string &status) > + { > + if (status.length() == 0) > + fprintf (stderr, "compacting table %s:\n", table.c_str()); It's not an error, so stderr seems wrong. BR, Jani. > + else > + fprintf (stderr, " %s\n", status.c_str()); > + } > +}; > + > void > notmuch_database_close_compact (notmuch_database_t *notmuch) > { > void *local = talloc_new (NULL); > - Xapian::Compactor compactor; > + NotmuchCompactor compactor; > char *notmuch_path, *xapian_path, *compact_xapian_path, *old_xapian_path; > > #if HAVE_XAPIAN_COMPACT > -- > 1.7.10.4 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 3/3] Add notmuch compact command 2012-10-17 15:28 [PATCH] notmuch compact support Ben Gamari 2012-10-17 15:28 ` [PATCH 1/3] Add notmuch_database_close_compact Ben Gamari 2012-10-17 15:28 ` [PATCH 2/3] Produce status messages during compacting Ben Gamari @ 2012-10-17 15:28 ` Ben Gamari 2012-10-17 19:07 ` Jani Nikula 2 siblings, 1 reply; 10+ messages in thread From: Ben Gamari @ 2012-10-17 15:28 UTC (permalink / raw) To: notmuch --- Makefile.local | 1 + notmuch-client.h | 3 +++ notmuch-compact.c | 43 +++++++++++++++++++++++++++++++++++++++++++ notmuch.c | 3 +++ 4 files changed, 50 insertions(+) create mode 100644 notmuch-compact.c diff --git a/Makefile.local b/Makefile.local index 7f2d4f1..e050ba6 100644 --- a/Makefile.local +++ b/Makefile.local @@ -258,6 +258,7 @@ notmuch_client_srcs = \ gmime-filter-headers.c \ hooks.c \ notmuch.c \ + notmuch-compact.c \ notmuch-config.c \ notmuch-count.c \ notmuch-dump.c \ diff --git a/notmuch-client.h b/notmuch-client.h index ae9344b..a6c624b 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -157,6 +157,9 @@ int notmuch_cat_command (void *ctx, int argc, char *argv[]); int +notmuch_compact_command (void *ctx, int argc, char *argv[]); + +int notmuch_config_command (void *ctx, int argc, char *argv[]); const char * diff --git a/notmuch-compact.c b/notmuch-compact.c new file mode 100644 index 0000000..6deb2ec --- /dev/null +++ b/notmuch-compact.c @@ -0,0 +1,43 @@ +/* notmuch - Not much of an email program, (just index and search) + * + * Copyright © 2009 Carl Worth + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see http://www.gnu.org/licenses/ . + * + * Author: Carl Worth <cworth@cworth.org> + */ + +#include "notmuch-client.h" + +int +notmuch_compact_command (void *ctx, unused (int argc), unused (char *argv[])) +{ + notmuch_config_t *config; + notmuch_database_t *notmuch; + + config = notmuch_config_open (ctx, NULL, NULL); + if (config == NULL) + return 1; + + if (notmuch_database_open (notmuch_config_get_database_path (config), + NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + return 1; + + printf ("Compacting database... "); + notmuch_database_close_compact (notmuch); + printf ("Done.\n"); + notmuch_database_destroy (notmuch); + + return 0; +} diff --git a/notmuch.c b/notmuch.c index 477a09c..7b6c5ae 100644 --- a/notmuch.c +++ b/notmuch.c @@ -74,6 +74,9 @@ static command_t commands[] = { { "restore", notmuch_restore_command, "[--accumulate] [<filename>]", "Restore the tags from the given dump file (see 'dump')." }, + { "compact", notmuch_compact_command, + NULL, + "Compacts the database." }, { "config", notmuch_config_command, "[get|set] <section>.<item> [value ...]", "Get or set settings in the notmuch configuration file." }, -- 1.7.10.4 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 3/3] Add notmuch compact command 2012-10-17 15:28 ` [PATCH 3/3] Add notmuch compact command Ben Gamari @ 2012-10-17 19:07 ` Jani Nikula 2012-10-18 6:29 ` Tomi Ollila 0 siblings, 1 reply; 10+ messages in thread From: Jani Nikula @ 2012-10-17 19:07 UTC (permalink / raw) To: Ben Gamari, notmuch Nag nag nag: Commit message. ;) The custom is to have a man page for each notmuch cli command. Small nitpicks below. BR, Jani. On Wed, 17 Oct 2012, Ben Gamari <bgamari.foss@gmail.com> wrote: > --- > Makefile.local | 1 + > notmuch-client.h | 3 +++ > notmuch-compact.c | 43 +++++++++++++++++++++++++++++++++++++++++++ > notmuch.c | 3 +++ > 4 files changed, 50 insertions(+) > create mode 100644 notmuch-compact.c > > diff --git a/Makefile.local b/Makefile.local > index 7f2d4f1..e050ba6 100644 > --- a/Makefile.local > +++ b/Makefile.local > @@ -258,6 +258,7 @@ notmuch_client_srcs = \ > gmime-filter-headers.c \ > hooks.c \ > notmuch.c \ > + notmuch-compact.c \ > notmuch-config.c \ > notmuch-count.c \ > notmuch-dump.c \ > diff --git a/notmuch-client.h b/notmuch-client.h > index ae9344b..a6c624b 100644 > --- a/notmuch-client.h > +++ b/notmuch-client.h > @@ -157,6 +157,9 @@ int > notmuch_cat_command (void *ctx, int argc, char *argv[]); > > int > +notmuch_compact_command (void *ctx, int argc, char *argv[]); > + > +int > notmuch_config_command (void *ctx, int argc, char *argv[]); > > const char * > diff --git a/notmuch-compact.c b/notmuch-compact.c > new file mode 100644 > index 0000000..6deb2ec > --- /dev/null > +++ b/notmuch-compact.c > @@ -0,0 +1,43 @@ > +/* notmuch - Not much of an email program, (just index and search) > + * > + * Copyright © 2009 Carl Worth It's your code, this year. > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 3 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see http://www.gnu.org/licenses/ . > + * > + * Author: Carl Worth <cworth@cworth.org> It's your code. > + */ > + > +#include "notmuch-client.h" > + > +int > +notmuch_compact_command (void *ctx, unused (int argc), unused (char *argv[])) > +{ > + notmuch_config_t *config; > + notmuch_database_t *notmuch; > + > + config = notmuch_config_open (ctx, NULL, NULL); > + if (config == NULL) > + return 1; > + > + if (notmuch_database_open (notmuch_config_get_database_path (config), > + NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) > + return 1; > + > + printf ("Compacting database... "); > + notmuch_database_close_compact (notmuch); > + printf ("Done.\n"); Or maybe not. Please add and check the return status of the compact call. > + notmuch_database_destroy (notmuch); > + > + return 0; > +} > diff --git a/notmuch.c b/notmuch.c > index 477a09c..7b6c5ae 100644 > --- a/notmuch.c > +++ b/notmuch.c > @@ -74,6 +74,9 @@ static command_t commands[] = { > { "restore", notmuch_restore_command, > "[--accumulate] [<filename>]", > "Restore the tags from the given dump file (see 'dump')." }, > + { "compact", notmuch_compact_command, > + NULL, > + "Compacts the database." }, > { "config", notmuch_config_command, > "[get|set] <section>.<item> [value ...]", > "Get or set settings in the notmuch configuration file." }, > -- > 1.7.10.4 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 3/3] Add notmuch compact command 2012-10-17 19:07 ` Jani Nikula @ 2012-10-18 6:29 ` Tomi Ollila 0 siblings, 0 replies; 10+ messages in thread From: Tomi Ollila @ 2012-10-18 6:29 UTC (permalink / raw) To: Jani Nikula, Ben Gamari, notmuch On Wed, Oct 17 2012, Jani Nikula <jani@nikula.org> wrote: > Nag nag nag: Commit message. ;) > > The custom is to have a man page for each notmuch cli command. > > Small nitpicks below. > > > BR, > Jani. > > > On Wed, 17 Oct 2012, Ben Gamari <bgamari.foss@gmail.com> wrote: >> --- >> Makefile.local | 1 + >> notmuch-client.h | 3 +++ >> notmuch-compact.c | 43 +++++++++++++++++++++++++++++++++++++++++++ >> notmuch.c | 3 +++ >> 4 files changed, 50 insertions(+) >> create mode 100644 notmuch-compact.c >> >> diff --git a/Makefile.local b/Makefile.local >> index 7f2d4f1..e050ba6 100644 >> --- a/Makefile.local >> +++ b/Makefile.local >> @@ -258,6 +258,7 @@ notmuch_client_srcs = \ >> gmime-filter-headers.c \ >> hooks.c \ >> notmuch.c \ >> + notmuch-compact.c \ >> notmuch-config.c \ >> notmuch-count.c \ >> notmuch-dump.c \ >> diff --git a/notmuch-client.h b/notmuch-client.h >> index ae9344b..a6c624b 100644 >> --- a/notmuch-client.h >> +++ b/notmuch-client.h >> @@ -157,6 +157,9 @@ int >> notmuch_cat_command (void *ctx, int argc, char *argv[]); >> >> int >> +notmuch_compact_command (void *ctx, int argc, char *argv[]); >> + >> +int >> notmuch_config_command (void *ctx, int argc, char *argv[]); >> >> const char * >> diff --git a/notmuch-compact.c b/notmuch-compact.c >> new file mode 100644 >> index 0000000..6deb2ec >> --- /dev/null >> +++ b/notmuch-compact.c >> @@ -0,0 +1,43 @@ >> +/* notmuch - Not much of an email program, (just index and search) >> + * >> + * Copyright © 2009 Carl Worth > > It's your code, this year. ... but you can give copyright to Carl if you wish... >> + * >> + * This program is free software: you can redistribute it and/or modify >> + * it under the terms of the GNU General Public License as published by >> + * the Free Software Foundation, either version 3 of the License, or >> + * (at your option) any later version. >> + * >> + * This program is distributed in the hope that it will be useful, >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> + * GNU General Public License for more details. >> + * >> + * You should have received a copy of the GNU General Public License >> + * along with this program. If not, see http://www.gnu.org/licenses/ . >> + * >> + * Author: Carl Worth <cworth@cworth.org> > > It's your code. Yes, all the praise and blames ;) ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH RFC?] Compactification support @ 2012-08-20 15:31 Ben Gamari 2012-08-20 15:31 ` [PATCH 3/3] Add notmuch compact command Ben Gamari 0 siblings, 1 reply; 10+ messages in thread From: Ben Gamari @ 2012-08-20 15:31 UTC (permalink / raw) To: notmuch Here is a small patchset adding support for database compaction when supported by Xapian. Here I add a function which both compacts and closes the database. This is due to it being not entirely clear whether one is allowed to compact the database while it is being held open for read/write. Since it is nice to be able to hold the read/write lock while compacting, I close the database only after the compaction has finished which Xapian seems to be fine with. That being said, the database is moved after compaction, so it seems appropriate to lump compaction in with closing the database. Lastly, I wasn't entirely sure whether/how Xapian's status messages should be presented to the user. At the moment I simply spit them out on stderr. Perhaps I should just pass a callback to notmuch_database_close_compact and leave this to the caller? Cheers, - Ben ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 3/3] Add notmuch compact command 2012-08-20 15:31 [PATCH RFC?] Compactification support Ben Gamari @ 2012-08-20 15:31 ` Ben Gamari 0 siblings, 0 replies; 10+ messages in thread From: Ben Gamari @ 2012-08-20 15:31 UTC (permalink / raw) To: notmuch --- Makefile.local | 1 + notmuch-client.h | 3 +++ notmuch-compact.c | 43 +++++++++++++++++++++++++++++++++++++++++++ notmuch.c | 3 +++ 4 files changed, 50 insertions(+) create mode 100644 notmuch-compact.c diff --git a/Makefile.local b/Makefile.local index de984ab..848605d 100644 --- a/Makefile.local +++ b/Makefile.local @@ -278,6 +278,7 @@ notmuch_client_srcs = \ gmime-filter-headers.c \ hooks.c \ notmuch.c \ + notmuch-compact.c \ notmuch-config.c \ notmuch-count.c \ notmuch-dump.c \ diff --git a/notmuch-client.h b/notmuch-client.h index ae9344b..a6c624b 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -157,6 +157,9 @@ int notmuch_cat_command (void *ctx, int argc, char *argv[]); int +notmuch_compact_command (void *ctx, int argc, char *argv[]); + +int notmuch_config_command (void *ctx, int argc, char *argv[]); const char * diff --git a/notmuch-compact.c b/notmuch-compact.c new file mode 100644 index 0000000..6deb2ec --- /dev/null +++ b/notmuch-compact.c @@ -0,0 +1,43 @@ +/* notmuch - Not much of an email program, (just index and search) + * + * Copyright © 2009 Carl Worth + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see http://www.gnu.org/licenses/ . + * + * Author: Carl Worth <cworth@cworth.org> + */ + +#include "notmuch-client.h" + +int +notmuch_compact_command (void *ctx, unused (int argc), unused (char *argv[])) +{ + notmuch_config_t *config; + notmuch_database_t *notmuch; + + config = notmuch_config_open (ctx, NULL, NULL); + if (config == NULL) + return 1; + + if (notmuch_database_open (notmuch_config_get_database_path (config), + NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + return 1; + + printf ("Compacting database... "); + notmuch_database_close_compact (notmuch); + printf ("Done.\n"); + notmuch_database_destroy (notmuch); + + return 0; +} diff --git a/notmuch.c b/notmuch.c index 477a09c..7b6c5ae 100644 --- a/notmuch.c +++ b/notmuch.c @@ -74,6 +74,9 @@ static command_t commands[] = { { "restore", notmuch_restore_command, "[--accumulate] [<filename>]", "Restore the tags from the given dump file (see 'dump')." }, + { "compact", notmuch_compact_command, + NULL, + "Compacts the database." }, { "config", notmuch_config_command, "[get|set] <section>.<item> [value ...]", "Get or set settings in the notmuch configuration file." }, -- 1.7.9.5 ^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2012-10-18 6:44 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-10-17 15:28 [PATCH] notmuch compact support Ben Gamari 2012-10-17 15:28 ` [PATCH 1/3] Add notmuch_database_close_compact Ben Gamari 2012-10-17 18:56 ` Jani Nikula 2012-10-18 6:44 ` Tomi Ollila 2012-10-17 15:28 ` [PATCH 2/3] Produce status messages during compacting Ben Gamari 2012-10-17 18:59 ` Jani Nikula 2012-10-17 15:28 ` [PATCH 3/3] Add notmuch compact command Ben Gamari 2012-10-17 19:07 ` Jani Nikula 2012-10-18 6:29 ` Tomi Ollila -- strict thread matches above, loose matches on Subject: below -- 2012-08-20 15:31 [PATCH RFC?] Compactification support Ben Gamari 2012-08-20 15:31 ` [PATCH 3/3] Add notmuch compact command Ben Gamari
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).