Fixes for building with -DDEBUG

Fixes for building with -DDEBUG

[David Bremner]

The first patch is an attempt at fixing the build failure in the ruby
bindings. I'm not sure if this is an acceptable use of "env" or not. I
noticed that all of our shebangs use env, but I guess that is optional
in some sense.

The remaining patches keep debugging output from disrupting the test
suite. Apparently only thread debugging output is problematic
currently.

[PATCH 1/4] bindings/ruby: unexport CFLAGS when building

[David Bremner]

This prevents breaking the ruby build when passing CFLAGS to other
parts of the build.
---
 bindings/Makefile.local | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bindings/Makefile.local b/bindings/Makefile.local
index bc960bbc..8e3cd051 100644
--- a/bindings/Makefile.local
+++ b/bindings/Makefile.local
@@ -10,7 +10,7 @@ ifeq ($(HAVE_RUBY_DEV),1)
 		LIBNOTMUCH="../../lib/$(LINKER_NAME)" \
 		NOTMUCH_SRCDIR='$(NOTMUCH_SRCDIR)' \
 		$(RUBY) extconf.rb --vendor
-	$(MAKE) -C $(dir)/ruby
+	env -u CFLAGS $(MAKE) -C $(dir)/ruby
 endif
 
 python-cffi-bindings: lib/$(LINKER_NAME)
-- 
2.30.2

[PATCH 2/4] lib/thread: add common prefix to debug messages, join lines

[David Bremner]

This will simplify filtering these message, e.g. in the test suite.
---
 lib/thread.cc | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index 46a50e80..5ac0db6f 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -25,7 +25,7 @@
 #include <glib.h>                                       /* GHashTable */
 
 #ifdef DEBUG_THREADING
-#define THREAD_DEBUG(format, ...) fprintf (stderr, format " (%s).\n", ##__VA_ARGS__, __location__)
+#define THREAD_DEBUG(format, ...) fprintf (stderr, "DT: " format " (%s).\n", ##__VA_ARGS__, __location__)
 #else
 #define THREAD_DEBUG(format, ...) do {} while (0)       /* ignored */
 #endif
@@ -400,7 +400,7 @@ _parent_via_in_reply_to (notmuch_thread_t *thread, notmuch_message_t *message)
     const char *in_reply_to;
 
     in_reply_to = _notmuch_message_get_in_reply_to (message);
-    THREAD_DEBUG ("checking message = %s in_reply_to=%s\n",
+    THREAD_DEBUG ("checking message = %s in_reply_to=%s",
 		  notmuch_message_get_message_id (message), in_reply_to);
 
     if (in_reply_to && (! EMPTY_STRING (in_reply_to)) &&
@@ -423,31 +423,31 @@ _parent_or_toplevel (notmuch_thread_t *thread, notmuch_message_t *message)
     const notmuch_string_list_t *references =
 	_notmuch_message_get_references (message);
 
-    THREAD_DEBUG ("trying to reparent via references: %s\n",
+    THREAD_DEBUG ("trying to reparent via references: %s",
 		  notmuch_message_get_message_id (message));
 
     for (notmuch_string_node_t *ref_node = references->head;
 	 ref_node; ref_node = ref_node->next) {
-	THREAD_DEBUG ("checking reference=%s\n", ref_node->string);
+	THREAD_DEBUG ("checking reference=%s", ref_node->string);
 	if ((g_hash_table_lookup_extended (thread->message_hash,
 					   ref_node->string, NULL,
 					   (void **) &new_parent))) {
 	    size_t new_depth = _notmuch_message_get_thread_depth (new_parent);
-	    THREAD_DEBUG ("got depth %lu\n", new_depth);
+	    THREAD_DEBUG ("got depth %lu", new_depth);
 	    if (new_depth > max_depth || ! parent) {
-		THREAD_DEBUG ("adding at depth %lu parent=%s\n", new_depth, ref_node->string);
+		THREAD_DEBUG ("adding at depth %lu parent=%s", new_depth, ref_node->string);
 		max_depth = new_depth;
 		parent = new_parent;
 	    }
 	}
     }
     if (parent) {
-	THREAD_DEBUG ("adding reply %s to parent=%s\n",
+	THREAD_DEBUG ("adding reply %s to parent=%s",
 		      notmuch_message_get_message_id (message),
 		      notmuch_message_get_message_id (parent));
 	_notmuch_message_add_reply (parent, message);
     } else {
-	THREAD_DEBUG ("adding as toplevel %s\n",
+	THREAD_DEBUG ("adding as toplevel %s",
 		      notmuch_message_get_message_id (message));
 	_notmuch_message_list_add_message (thread->toplevel_list, message);
     }
@@ -482,13 +482,13 @@ _resolve_thread_relationships (notmuch_thread_t *thread)
      */
     if (first_node) {
 	message = first_node->message;
-	THREAD_DEBUG ("checking first message  %s\n",
+	THREAD_DEBUG ("checking first message  %s",
 		      notmuch_message_get_message_id (message));
 
 	if (_notmuch_message_list_empty (maybe_toplevel_list) ||
 	    ! _parent_via_in_reply_to (thread, message)) {
 
-	    THREAD_DEBUG ("adding first message as toplevel = %s\n",
+	    THREAD_DEBUG ("adding first message as toplevel = %s",
 			  notmuch_message_get_message_id (message));
 	    _notmuch_message_list_add_message (maybe_toplevel_list, message);
 	}
-- 
2.30.2

[PATCH 3/4] test: ignore debugging messages

[David Bremner]

Previously building with "-DDEBUG" broke the test suite in several places.
---
 test/T300-encoding.sh   | 12 ++++++------
 test/T310-emacs.sh      |  1 -
 test/T450-emacs-show.sh |  1 -
 test/test-lib.sh        | 23 +++++++++++++++--------
 4 files changed, 21 insertions(+), 16 deletions(-)

diff --git a/test/T300-encoding.sh b/test/T300-encoding.sh
index 1e9d2a3d..6fcd10c5 100755
--- a/test/T300-encoding.sh
+++ b/test/T300-encoding.sh
@@ -31,18 +31,18 @@ test_expect_equal "$output" "thread:0000000000000002   2001-01-05 [1/1] Notmuch
 
 test_begin_subtest "RFC 2047 encoded word with spaces"
 add_message '[subject]="=?utf-8?q?encoded word with spaces?="'
-output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_show_sanitize)
-test_expect_equal "$output" "thread:0000000000000003   2001-01-05 [1/1] Notmuch Test Suite; encoded word with spaces (inbox unread)"
+output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; encoded word with spaces (inbox unread)"
 
 test_begin_subtest "RFC 2047 encoded words back to back"
 add_message '[subject]="=?utf-8?q?encoded-words-back?==?utf-8?q?to-back?="'
-output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_show_sanitize)
-test_expect_equal "$output" "thread:0000000000000004   2001-01-05 [1/1] Notmuch Test Suite; encoded-words-backto-back (inbox unread)"
+output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; encoded-words-backto-back (inbox unread)"
 
 test_begin_subtest "RFC 2047 encoded words without space before or after"
 add_message '[subject]="=?utf-8?q?encoded?=word without=?utf-8?q?space?=" '
-output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_show_sanitize)
-test_expect_equal "$output" "thread:0000000000000005   2001-01-05 [1/1] Notmuch Test Suite; encodedword withoutspace (inbox unread)"
+output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; encodedword withoutspace (inbox unread)"
 
 test_begin_subtest "Mislabeled Windows-1252 encoding"
 add_message '[content-type]="text/plain; charset=iso-8859-1"'                           \
diff --git a/test/T310-emacs.sh b/test/T310-emacs.sh
index 851ef64e..dad09b11 100755
--- a/test/T310-emacs.sh
+++ b/test/T310-emacs.sh
@@ -1058,7 +1058,6 @@ End of search results.
 === MESSAGES ===
 YYY/notmuch_fail exited with status 1 (see *Notmuch errors* for more details)
 === ERROR ===
-[XXX]
 YYY/notmuch_fail exited with status 1
 command: YYY/notmuch_fail search --format\=sexp --format-version\=4 --sort\=newest-first tag\:inbox
 exit status: 1"
diff --git a/test/T450-emacs-show.sh b/test/T450-emacs-show.sh
index bd76d378..52353bf4 100755
--- a/test/T450-emacs-show.sh
+++ b/test/T450-emacs-show.sh
@@ -190,7 +190,6 @@ test_expect_equal "$(notmuch_emacs_error_sanitize notmuch_fail OUTPUT MESSAGES E
 === MESSAGES ===
 This is an error (see *Notmuch errors* for more details)
 === ERROR ===
-[XXX]
 This is an error
 command: YYY/notmuch_fail show --format\\=sexp --format-version\\=4 --decrypt\\=true --exclude\\=false \\' \\* \\'
 exit status: 1
diff --git a/test/test-lib.sh b/test/test-lib.sh
index d46bb4c3..69d9583c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -637,6 +637,11 @@ print(msg.as_string(False))
 ' "$@"
 }
 
+notmuch_debug_sanitize ()
+{
+    grep -v '^D.:'
+}
+
 notmuch_exception_sanitize ()
 {
     perl -pe 's/(A Xapian exception occurred at .*[.]cc?):([0-9]*)/\1:XXX/'
@@ -644,7 +649,7 @@ notmuch_exception_sanitize ()
 
 notmuch_search_sanitize ()
 {
-    perl -pe 's/("?thread"?: ?)("?)................("?)/\1\2XXX\3/'
+    notmuch_debug_sanitize | perl -pe 's/("?thread"?: ?)("?)................("?)/\1\2XXX\3/'
 }
 
 notmuch_search_files_sanitize ()
@@ -664,10 +669,11 @@ notmuch_show_sanitize ()
 }
 notmuch_show_sanitize_all ()
 {
-    sed \
-	-e 's| filename:.*| filename:XXXXX|' \
-	-e 's| id:[^ ]* | id:XXXXX |' | \
-	notmuch_date_sanitize
+    notmuch_debug_sanitize |
+	sed \
+	    -e 's| filename:.*| filename:XXXXX|' \
+	    -e 's| id:[^ ]* | id:XXXXX |' | \
+            notmuch_date_sanitize
 }
 
 notmuch_json_show_sanitize ()
@@ -688,9 +694,10 @@ notmuch_emacs_error_sanitize ()
     shift
     for file in "$@"; do
 	echo "=== $file ==="
-	cat "$file"
+	notmuch_debug_sanitize < "$file"
     done | sed \
-	-e 's/^\[.*\]$/[XXX]/' \
+	-e '/^$/d' \
+	-e '/^\[.*\]$/d' \
 	-e "s|^\(command: \)\{0,1\}/.*/$command|\1YYY/$command|"
 }
 
@@ -1133,7 +1140,7 @@ test_C () {
     echo "== stdout ==" > OUTPUT.stdout
     echo "== stderr ==" > OUTPUT.stderr
     ./${exec_file} "$@" 1>>OUTPUT.stdout 2>>OUTPUT.stderr
-    notmuch_dir_sanitize OUTPUT.stdout OUTPUT.stderr | notmuch_exception_sanitize > OUTPUT
+    notmuch_dir_sanitize OUTPUT.stdout OUTPUT.stderr | notmuch_exception_sanitize | notmuch_debug_sanitize > OUTPUT
 }
 
 make_shim () {
-- 
2.30.2

[PATCH 4/4] test: quiet some extra debugging output

[David Bremner]

This output does not cause test failures, but may make it harder to
interpret the output.
---
 test/T140-excludes.sh    | 2 +-
 test/T380-atomicity.sh   | 2 +-
 test/T700-reindex.sh     | 4 ++--
 test/T750-user-header.sh | 4 ++--
 4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index acab5381..7b5bea2e 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -22,7 +22,7 @@ generate_thread ()
     done
     notmuch new > /dev/null
     # We cannot retrieve the thread_id until after we have run notmuch new.
-    gen_thread_id=`notmuch search --output=threads id:${gen_thread_msg_id[0]}`
+    gen_thread_id=$(notmuch search --output=threads id:${gen_thread_msg_id[0]} 2>/dev/null)
 }
 
 #############################################
diff --git a/test/T380-atomicity.sh b/test/T380-atomicity.sh
index 45de2228..afe49d93 100755
--- a/test/T380-atomicity.sh
+++ b/test/T380-atomicity.sh
@@ -67,7 +67,7 @@ if test_require_external_prereq gdb; then
     ${TEST_GDB} -tty /dev/null -batch -x $NOTMUCH_SRCDIR/test/atomicity.py notmuch 1>gdb.out 2>&1
 
     # Get the final, golden output
-    notmuch search '*' > expected
+    notmuch search '*' 2>/dev/null > expected
 
     # Check output against golden output
     outcount=$(cat outcount)
diff --git a/test/T700-reindex.sh b/test/T700-reindex.sh
index f51130e8..52bba4d3 100755
--- a/test/T700-reindex.sh
+++ b/test/T700-reindex.sh
@@ -6,8 +6,8 @@ add_email_corpus
 
 notmuch tag +usertag1 '*'
 
-notmuch search '*' | notmuch_search_sanitize > initial-threads
-notmuch search --output=messages '*' > initial-message-ids
+notmuch search '*' 2>1 | notmuch_search_sanitize > initial-threads
+notmuch search --output=messages '*' 2>/dev/null > initial-message-ids
 notmuch dump > initial-dump
 
 test_begin_subtest 'reindex preserves threads'
diff --git a/test/T750-user-header.sh b/test/T750-user-header.sh
index 05f80885..03c43656 100755
--- a/test/T750-user-header.sh
+++ b/test/T750-user-header.sh
@@ -4,8 +4,8 @@ test_description='indexing user specified headers'
 
 add_email_corpus
 
-notmuch search '*' | notmuch_search_sanitize > initial-threads
-notmuch search --output=messages '*' > initial-message-ids
+notmuch search '*' 2>1 | notmuch_search_sanitize > initial-threads
+notmuch search --output=messages '*' 2>/dev/null > initial-message-ids
 notmuch dump > initial-dump
 
 test_begin_subtest "adding illegal prefix name, bad utf8"
-- 
2.30.2

Re: Fixes for building with -DDEBUG

[David Bremner]

David Bremner <david@tethera.net> writes:

> The first patch is an attempt at fixing the build failure in the ruby
> bindings. I'm not sure if this is an acceptable use of "env" or not. I
> noticed that all of our shebangs use env, but I guess that is optional
> in some sense.

I belatedly checked, and POSIX env does not have -u

Re: [PATCH 1/4] bindings/ruby: unexport CFLAGS when building

[Tomi Ollila]

On Sun, May 16 2021, David Bremner wrote:

> This prevents breaking the ruby build when passing CFLAGS to other
> parts of the build.
> ---
>  bindings/Makefile.local | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/bindings/Makefile.local b/bindings/Makefile.local
> index bc960bbc..8e3cd051 100644
> --- a/bindings/Makefile.local
> +++ b/bindings/Makefile.local
> @@ -10,7 +10,7 @@ ifeq ($(HAVE_RUBY_DEV),1)
>  		LIBNOTMUCH="../../lib/$(LINKER_NAME)" \
>  		NOTMUCH_SRCDIR='$(NOTMUCH_SRCDIR)' \
>  		$(RUBY) extconf.rb --vendor
> -	$(MAKE) -C $(dir)/ruby
> +	env -u CFLAGS $(MAKE) -C $(dir)/ruby

As you mentioned posix env does not have -u (that's shame),
perhaps CFLAGS= $(MAKE) -C $(dir)/ruby works -- that is slightly
different as now CFLAGS is defined in env, just empty.

Tomi

>  endif
>  
>  python-cffi-bindings: lib/$(LINKER_NAME)
> -- 
> 2.30.2
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org

Re: Fixes for building with -DDEBUG

[David Bremner]

David Bremner <david@tethera.net> writes:

> The first patch is an attempt at fixing the build failure in the ruby
> bindings. I'm not sure if this is an acceptable use of "env" or not. I
> noticed that all of our shebangs use env, but I guess that is optional
> in some sense.
>
> The remaining patches keep debugging output from disrupting the test
> suite. Apparently only thread debugging output is problematic
> currently.

2-4 applied to master

d