memory leak cleanup for notmuch show

memory leak cleanup for notmuch show

[David Bremner]

Inspired by Jeff's patch, I updated the memory test suite to test
notmuch-show, this series is the result. I also include Jeff's
original patch in this series.

[PATCH 1/6] perf-test: use 'eval' in memory_run

[David Bremner]

This allows the use of redirection in the tests
---
 performance-test/perf-test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh
index 00d2f1c6..c89d5aab 100644
--- a/performance-test/perf-test-lib.sh
+++ b/performance-test/perf-test-lib.sh
@@ -149,7 +149,7 @@ memory_run ()
 
     printf "[ %d ]\t%s\n" $test_count "$1"
 
-    NOTMUCH_TALLOC_REPORT="$talloc_log" valgrind --leak-check=full --log-file="$log_file" $2
+    NOTMUCH_TALLOC_REPORT="$talloc_log" eval "valgrind --leak-check=full --log-file='$log_file' $2"
 
     awk '/LEAK SUMMARY/,/suppressed/ { sub(/^==[0-9]*==/," "); print }' "$log_file"
     echo
-- 
2.11.0

[PATCH 2/6] perf-test: add simple memory tests for notmuch-show

[David Bremner]

These are probably too slow to run with the full corpus
---
 performance-test/M02-show.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100755 performance-test/M02-show.sh

diff --git a/performance-test/M02-show.sh b/performance-test/M02-show.sh
new file mode 100755
index 00000000..d73035ea
--- /dev/null
+++ b/performance-test/M02-show.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+test_description='show'
+
+. ./perf-test-lib.sh || exit 1
+
+memory_start
+
+memory_run 'show *' "notmuch show '*' 1>/dev/null"
+memory_run 'show --format=json *' "notmuch show --format=json '*' 1>/dev/null"
+memory_run 'show --format=sexp *' "notmuch show --format=sexp '*' 1>/dev/null"
+
+memory_done
-- 
2.11.0

[PATCH 3/6] fix memory leaks in notmuch-show.c:format_headers_sprinter()

[David Bremner]

From: Jeffrey Stedfast <jestedfa@microsoft.com>

Internet_address_list_to_string() and
g_mime_message_get_date_as_string() return allocated string buffers
and not const, so from what I can tell from taking a look at the
sprinter-sexp.c’s sexp_string() function, the code leaks the
recipients_string as well as the date string.
---
 notmuch-show.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index aff93803..095595e2 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -202,8 +202,9 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
      * reflected in the file devel/schemata. */
 
     InternetAddressList *recipients;
-    const char *recipients_string;
+    char *recipients_string;
     const char *reply_to_string;
+    char *date_string;
 
     sp->begin_map (sp);
 
@@ -218,6 +219,7 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
     if (recipients_string) {
 	sp->map_key (sp, "To");
 	sp->string (sp, recipients_string);
+	g_free (recipients_string);
     }
 
     recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_CC);
@@ -225,6 +227,7 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
     if (recipients_string) {
 	sp->map_key (sp, "Cc");
 	sp->string (sp, recipients_string);
+	g_free (recipients_string);
     }
 
     recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_BCC);
@@ -232,6 +235,7 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
     if (recipients_string) {
 	sp->map_key (sp, "Bcc");
 	sp->string (sp, recipients_string);
+	g_free (recipients_string);
     }
 
     reply_to_string = g_mime_message_get_reply_to (message);
@@ -248,7 +252,9 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
 	sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), "References"));
     } else {
 	sp->map_key (sp, "Date");
-	sp->string (sp, g_mime_message_get_date_as_string (message));
+	date_string = g_mime_message_get_date_as_string (message);
+	sp->string (sp, date_string);
+	g_free (date_string);
     }
 
     sp->end (sp);
-- 
2.11.0

[PATCH 4/6] cli/show: fix some memory leaks in format_part_text

[David Bremner]

Mimic Jeff Stedfast's changes to format_headers_sprinter, clean up use
of internet_address_list_to_string and
g_mime_message_get_date_as_string.
---
 notmuch-show.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 095595e2..b0afc29e 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -460,7 +460,8 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
     if (GMIME_IS_MESSAGE (node->part)) {
 	GMimeMessage *message = GMIME_MESSAGE (node->part);
 	InternetAddressList *recipients;
-	const char *recipients_string;
+	char *recipients_string;
+	char *date_string;
 
 	printf ("\fheader{\n");
 	if (node->envelope_file)
@@ -471,11 +472,15 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
 	recipients_string = internet_address_list_to_string (recipients, 0);
 	if (recipients_string)
 	    printf ("To: %s\n", recipients_string);
+	g_free (recipients_string);
 	recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_CC);
 	recipients_string = internet_address_list_to_string (recipients, 0);
 	if (recipients_string)
 	    printf ("Cc: %s\n", recipients_string);
-	printf ("Date: %s\n", g_mime_message_get_date_as_string (message));
+	g_free (recipients_string);
+	date_string = g_mime_message_get_date_as_string (message);
+	printf ("Date: %s\n", date_string);
+	g_free (date_string);
 	printf ("\fheader}\n");
 
 	printf ("\fbody{\n");
-- 
2.11.0

[PATCH 5/6] cli/show: fix usage of g_mime_content_type_to_string

[David Bremner]

It returns an "allocated string", which needs to be freed.
---
 notmuch-show.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index b0afc29e..43ee9021 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -438,6 +438,7 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
 		notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED) ? 1 : 0,
 		notmuch_message_get_filename (message));
     } else {
+	char *content_string;
 	const char *disposition = _get_disposition (meta);
 	const char *cid = g_mime_object_get_content_id (meta);
 	const char *filename = leaf ?
@@ -454,7 +455,10 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
 	    printf (", Filename: %s", filename);
 	if (cid)
 	    printf (", Content-id: %s", cid);
-	printf (", Content-type: %s\n", g_mime_content_type_to_string (content_type));
+
+	content_string = g_mime_content_type_to_string (content_type);
+	printf (", Content-type: %s\n", content_string);
+	g_free (content_string);
     }
 
     if (GMIME_IS_MESSAGE (node->part)) {
@@ -495,8 +499,9 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
 	    show_text_part_content (node->part, stream_stdout, 0);
 	    g_object_unref(stream_stdout);
 	} else {
-	    printf ("Non-text part: %s\n",
-		    g_mime_content_type_to_string (content_type));
+	    char *content_string = g_mime_content_type_to_string (content_type);
+	    printf ("Non-text part: %s\n", content_string);
+	    g_free (content_string);
 	}
     }
 
@@ -564,6 +569,7 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node,
     GMimeObject *meta = node->envelope_part ?
 	GMIME_OBJECT (node->envelope_part) : node->part;
     GMimeContentType *content_type = g_mime_object_get_content_type (meta);
+    char *content_string;
     const char *disposition = _get_disposition (meta);
     const char *cid = g_mime_object_get_content_id (meta);
     const char *filename = GMIME_IS_PART (node->part) ?
@@ -592,7 +598,9 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node,
     }
 
     sp->map_key (sp, "content-type");
-    sp->string (sp, g_mime_content_type_to_string (content_type));
+    content_string = g_mime_content_type_to_string (content_type);
+    sp->string (sp, content_string);
+    g_free (content_string);
 
     if (disposition) {
 	sp->map_key (sp, "content-disposition");
-- 
2.11.0

[PATCH 6/6] cli/show: unref crlf filter.

[David Bremner]

Mimic the handling of the other filter g_objects. This cleans up a
fair sized memory leak.
---
 notmuch-show.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 43ee9021..7451d5ab 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -276,6 +276,7 @@ show_text_part_content (GMimeObject *part, GMimeStream *stream_out,
 {
     GMimeContentType *content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
     GMimeStream *stream_filter = NULL;
+    GMimeFilter *crlf_filter = NULL;
     GMimeDataWrapper *wrapper;
     const char *charset;
 
@@ -287,8 +288,10 @@ show_text_part_content (GMimeObject *part, GMimeStream *stream_out,
 	return;
 
     stream_filter = g_mime_stream_filter_new (stream_out);
+    crlf_filter = g_mime_filter_crlf_new (FALSE, FALSE);
     g_mime_stream_filter_add(GMIME_STREAM_FILTER (stream_filter),
-			     g_mime_filter_crlf_new (FALSE, FALSE));
+			     crlf_filter);
+    g_object_unref (crlf_filter);
 
     charset = g_mime_object_get_content_type_parameter (part, "charset");
     if (charset) {
-- 
2.11.0

[PATCH] perf-test/mem: add simple memory tests for notmuch search

[David Bremner]

Just copy and replace from the show tests. Currently these show no
major leaks.
---
 performance-test/M03-search.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100755 performance-test/M03-search.sh

diff --git a/performance-test/M03-search.sh b/performance-test/M03-search.sh
new file mode 100755
index 00000000..8d026eee
--- /dev/null
+++ b/performance-test/M03-search.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+test_description='search'
+
+. ./perf-test-lib.sh || exit 1
+
+memory_start
+
+memory_run 'search *' "notmuch search '*' 1>/dev/null"
+memory_run 'search --format=json *' "notmuch search --format=json '*' 1>/dev/null"
+memory_run 'search --format=sexp *' "notmuch search --format=sexp '*' 1>/dev/null"
+
+memory_done
-- 
2.11.0

Re: [PATCH 1/6] perf-test: use 'eval' in memory_run

[Tomi Ollila]

On Sat, Mar 18 2017, David Bremner <david@tethera.net> wrote:

> This allows the use of redirection in the tests
> ---
>  performance-test/perf-test-lib.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh
> index 00d2f1c6..c89d5aab 100644
> --- a/performance-test/perf-test-lib.sh
> +++ b/performance-test/perf-test-lib.sh
> @@ -149,7 +149,7 @@ memory_run ()
>  
>      printf "[ %d ]\t%s\n" $test_count "$1"
>  
> -    NOTMUCH_TALLOC_REPORT="$talloc_log" valgrind --leak-check=full --log-file="$log_file" $2
> +    NOTMUCH_TALLOC_REPORT="$talloc_log" eval "valgrind --leak-check=full --log-file='$log_file' $2"

For the record, this would have worked w/o the double quotes (which I
thought would have not), but it is somewhat safer for someone to copy
this to some use. If there were literal '>'s in the line, then those
redirections would have been done in 'eval' (with these quotes). Without
quotes, '>' redirection would have been done before 'eval'. But 
when '>' is given in variable $2, the redirection would have been
done in eval (after shell has expanded the line for it) in any case.

all that said, this particular change OK...

Tomi

>  
>      awk '/LEAK SUMMARY/,/suppressed/ { sub(/^==[0-9]*==/," "); print }' "$log_file"
>      echo

Re: memory leak cleanup for notmuch show

[Tomi Ollila]

On Sat, Mar 18 2017, David Bremner <david@tethera.net> wrote:

> Inspired by Jeff's patch, I updated the memory test suite to test
> notmuch-show, this series is the result. I also include Jeff's
> original patch in this series.

series LGTM. tests pass (fedora 25, gmime 2.6.20, xapian 1.2.24)

Tomi

Re: memory leak cleanup for notmuch show

[David Bremner]

David Bremner <david@tethera.net> writes:

> Inspired by Jeff's patch, I updated the memory test suite to test
> notmuch-show, this series is the result. I also include Jeff's
> original patch in this series.
>

Series pushed,

d