[PATCH] perf-test: add option to run perf

[PATCH] perf-test: add option to run perf

[David Bremner]

Although the generation of perf data is not as slow as valgrind, it
seems simplest to re-use the machinery already there to save the logs
in a timestamped subdirectory.
---

I'm not sure about the use of --call-graph=lbr here. On the one hand,
the documentation mentions it depends on recent intel hardware. On the
other hand, fp does not work for me (just gives 00000000000) and dwarf
yields giant data files that take painfully long to run "perf report"
on.


 performance-test/perf-test-lib.sh | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)

diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh
index b70288cc..216d1824 100644
--- a/performance-test/perf-test-lib.sh
+++ b/performance-test/perf-test-lib.sh
@@ -2,6 +2,7 @@
 
 corpus_size=large
 
+use_perf=0
 while test "$#" -ne 0
 do
 	case "$1" in
@@ -9,6 +10,10 @@ do
 		debug=t;
 		shift
 		;;
+	-p|--perf)
+		use_perf=1;
+		shift
+		;;
 	-s|--small)
 		corpus_size=small;
 		shift
@@ -127,10 +132,20 @@ notmuch_new_with_cache ()
     fi
 }
 
+make_log_dir () {
+    local timestamp=$(date +%Y%m%dT%H%M%S)
+    log_dir="${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}"
+    mkdir -p ${log_dir}
+}
+
 time_start ()
 {
     add_email_corpus
 
+    if [[ "X$use_perf" = "X1" ]]; then
+	make_log_dir
+    fi
+
     print_header
 
     notmuch_new_with_cache time_run
@@ -140,9 +155,7 @@ memory_start ()
 {
     add_email_corpus
 
-    local timestamp=$(date +%Y%m%dT%H%M%S)
-    log_dir="${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}"
-    mkdir -p ${log_dir}
+    make_log_dir
 
     notmuch_new_with_cache memory_run
 }
@@ -193,7 +206,13 @@ time_run ()
     printf "  %-22s" "$1"
     test_count=$(($test_count+1))
     if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi
-    if ! eval >&3 "/usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' $2" ; then
+    if [[ "X$use_perf" = "X1" ]]; then
+	command_str="perf record --call-graph=lbr -o ${log_dir}/${test_count}.perf $2"
+    else
+	command_str="/usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' $2"
+    fi
+
+    if ! eval >&3 "$command_str" ; then
 	test_failure=$(($test_failure + 1))
 	return 1
     fi
-- 
2.30.2

Re: [PATCH] perf-test: add option to run perf

[Tomi Ollila]

On Sun, Apr 25 2021, David Bremner wrote:

> Although the generation of perf data is not as slow as valgrind, it
> seems simplest to re-use the machinery already there to save the logs
> in a timestamped subdirectory.
> ---
>
> I'm not sure about the use of --call-graph=lbr here. On the one hand,
> the documentation mentions it depends on recent intel hardware. On the
> other hand, fp does not work for me (just gives 00000000000) and dwarf
> yields giant data files that take painfully long to run "perf report"
> on.
>
>
>  performance-test/perf-test-lib.sh | 27 +++++++++++++++++++++++----
>  1 file changed, 23 insertions(+), 4 deletions(-)
>
> diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh
> index b70288cc..216d1824 100644
> --- a/performance-test/perf-test-lib.sh
> +++ b/performance-test/perf-test-lib.sh
> @@ -2,6 +2,7 @@
>  
>  corpus_size=large
>  
> +use_perf=0
>  while test "$#" -ne 0

In above, newline consistency (perhaps all vars in "block", then
newline). Now that this is added perhaps debug also (or is it
supposed that debug=t from environment has effect ;)

>  do
>  	case "$1" in
> @@ -9,6 +10,10 @@ do
>  		debug=t;
>  		shift
>  		;;
> +	-p|--perf)
> +		use_perf=1;
> +		shift
> +		;;
>  	-s|--small)
>  		corpus_size=small;
>  		shift
> @@ -127,10 +132,20 @@ notmuch_new_with_cache ()
>      fi
>  }
>  
> +make_log_dir () {
> +    local timestamp=$(date +%Y%m%dT%H%M%S)
> +    log_dir="${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}"
> +    mkdir -p ${log_dir}

This is copy from deleted code -- just that quoting is just as "wrong" as
it can be >;) -- var=val does not need quotes, OTOH "val" in command
argument does need quotes (in order to escape $IFS splitting) -- if that
mattered.

> +}
> +
>  time_start ()
>  {
>      add_email_corpus
>  
> +    if [[ "X$use_perf" = "X1" ]]; then

Consistently w/  [[ "$corpus_size" != "large" ]] just if [[ "$use_perf" = 1 ]]

> +	make_log_dir
> +    fi
> +
>      print_header
>  
>      notmuch_new_with_cache time_run
> @@ -140,9 +155,7 @@ memory_start ()
>  {
>      add_email_corpus
>  
> -    local timestamp=$(date +%Y%m%dT%H%M%S)
> -    log_dir="${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}"
> -    mkdir -p ${log_dir}
> +    make_log_dir
>  
>      notmuch_new_with_cache memory_run
>  }
> @@ -193,7 +206,13 @@ time_run ()
>      printf "  %-22s" "$1"
>      test_count=$(($test_count+1))
>      if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi
> -    if ! eval >&3 "/usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' $2" ; then
> +    if [[ "X$use_perf" = "X1" ]]; then

ditto

> +	command_str="perf record --call-graph=lbr -o ${log_dir}/${test_count}.perf $2"
> +    else
> +	command_str="/usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' $2"
> +    fi
> +
> +    if ! eval >&3 "$command_str" ; then
>  	test_failure=$(($test_failure + 1))
>  	return 1
>      fi
> -- 
> 2.30.2

Re: [PATCH] perf-test: add option to run perf

[David Bremner]

David Bremner <david@tethera.net> writes:

> Although the generation of perf data is not as slow as valgrind, it
> seems simplest to re-use the machinery already there to save the logs
> in a timestamped subdirectory.
> ---

applied a version updated with Tomi's suggestions and an extra option
for setting the call-graph type.