[PATCH] test: Produce useful output when the HTML with images test fails

[PATCH] test: Produce useful output when the HTML with images test fails

[Austin Clements]

Previously, this would simply indicate that the grep failed without
any indication of the Emacs output it failed on.  Now we take
advantage of the test framework's handling of stdout to display the
incorrect Emacs output if the test fails.
---

Besides being a good idea in general, this is the first step toward
fixing this test on the Notmuch buildbot, since right now there's no
indication of why it's failing.

 test/emacs |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/test/emacs b/test/emacs
index 77265b0..5649d55 100755
--- a/test/emacs
+++ b/test/emacs
@@ -809,9 +809,10 @@ test_emacs "(let ((mm-text-html-renderer
 	      (notmuch-show \"id:${gen_msg_id}\"))
 	    (test-output)" > /dev/null
 # Different Emacs versions and renderers give very different results,
-# so just check that something reasonable showed up.
+# so just check that something reasonable showed up.  We first cat the
+# output so the test framework will print it if the test fails.
 test_expect_success "Rendering HTML mail with images" \
-    'grep -q smiley OUTPUT'
+    'cat OUTPUT && grep -q smiley OUTPUT'
 
 
 test_done
-- 
1.7.10.4

Re: [PATCH] test: Produce useful output when the HTML with images test fails

[Tomi Ollila]

On Tue, Nov 20 2012, Austin Clements <amdragon@MIT.EDU> wrote:

> Previously, this would simply indicate that the grep failed without
> any indication of the Emacs output it failed on.  Now we take
> advantage of the test framework's handling of stdout to display the
> incorrect Emacs output if the test fails.
> ---
>
> Besides being a good idea in general, this is the first step toward
> fixing this test on the Notmuch buildbot, since right now there's no
> indication of why it's failing.

Tom: could you do separate test on the buildbot machine (master-emacs24
config) with this patch applied and then running
NOTMUCH_TESTS=emacs make test

Tomi

>  test/emacs |    5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/test/emacs b/test/emacs
> index 77265b0..5649d55 100755
> --- a/test/emacs
> +++ b/test/emacs
> @@ -809,9 +809,10 @@ test_emacs "(let ((mm-text-html-renderer
>  	      (notmuch-show \"id:${gen_msg_id}\"))
>  	    (test-output)" > /dev/null
>  # Different Emacs versions and renderers give very different results,
> -# so just check that something reasonable showed up.
> +# so just check that something reasonable showed up.  We first cat the
> +# output so the test framework will print it if the test fails.
>  test_expect_success "Rendering HTML mail with images" \
> -    'grep -q smiley OUTPUT'
> +    'cat OUTPUT && grep -q smiley OUTPUT'
>  
>  
>  test_done
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: Produce useful output when the HTML with images test fails

[Tomi Ollila]

On Wed, Nov 21 2012, Tomi Ollila <tomi.ollila@iki.fi> wrote:

> On Tue, Nov 20 2012, Austin Clements <amdragon@MIT.EDU> wrote:
>
>> Previously, this would simply indicate that the grep failed without
>> any indication of the Emacs output it failed on.  Now we take
>> advantage of the test framework's handling of stdout to display the
>> incorrect Emacs output if the test fails.
>> ---
>>
>> Besides being a good idea in general, this is the first step toward
>> fixing this test on the Notmuch buildbot, since right now there's no
>> indication of why it's failing.
>
> Tom: could you do separate test on the buildbot machine (master-emacs24
> config) with this patch applied and then running
> NOTMUCH_TESTS=emacs make test

Hmm, I just remembered something:

From notmuch IRC log Thu Oct 25 2012

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
< Domo_> For me the 'shr test fails by missing libxml-parse-html-region ...
< Domo_> which is defined in src/xml.c -- and exists by default on 
         another emacs...
< Domo_> which probably means that my self-compiled emacs24 just doesn't 
         have that xml code compiled in :(
< Domo_> #ifdef HAVE_LIBXML2
< Domo_> ...
< Domo_> #endif /* HAVE_LIBXML2 */
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

So, this is something to be checked for -- installing 
libxml2-devel-2.7.6-8.el6_3.3.x86_64 (on Scientific Linux 6 machine)
and rebuilding emacs made things work for me.

I'd guess the recipe to build emacs 24 on gentoo had this dependency
but maybe emacs24 is so new that at the time buildbot machine got
emacs24 that wasn't there yet ???

Tomi

>
> Tomi
>
>>  test/emacs |    5 +++--
>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/test/emacs b/test/emacs
>> index 77265b0..5649d55 100755
>> --- a/test/emacs
>> +++ b/test/emacs
>> @@ -809,9 +809,10 @@ test_emacs "(let ((mm-text-html-renderer
>>  	      (notmuch-show \"id:${gen_msg_id}\"))
>>  	    (test-output)" > /dev/null
>>  # Different Emacs versions and renderers give very different results,
>> -# so just check that something reasonable showed up.
>> +# so just check that something reasonable showed up.  We first cat the
>> +# output so the test framework will print it if the test fails.
>>  test_expect_success "Rendering HTML mail with images" \
>> -    'grep -q smiley OUTPUT'
>> +    'cat OUTPUT && grep -q smiley OUTPUT'
>>  
>>  
>>  test_done
>> -- 
>> 1.7.10.4
>>
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: Produce useful output when the HTML with images test fails

[Tomi Ollila]

On Wed, Nov 21 2012, Tomi Ollila <tomi.ollila@iki.fi> wrote:
>
> From notmuch IRC log Thu Oct 25 2012
>
> --8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
> < Domo_> For me the 'shr test fails by missing libxml-parse-html-region ...
> < Domo_> which is defined in src/xml.c -- and exists by default on 
>          another emacs...
> < Domo_> which probably means that my self-compiled emacs24 just doesn't 
>          have that xml code compiled in :(
> < Domo_> #ifdef HAVE_LIBXML2
> < Domo_> ...
> < Domo_> #endif /* HAVE_LIBXML2 */
> --8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

That's easy to test:

run emacs24

Enter

M-x describe-function libxml-parse-html-region

to see whether the function is defined.

The output should look like:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
libxml-parse-html-region is a built-in function in `C source code'.

(libxml-parse-html-region START END &optional BASE-URL)

Parse the region as an HTML document and return the parse tree.
If BASE-URL is non-nil, it is used to expand relative URLs.
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

Tomi

Re: [PATCH] test: Produce useful output when the HTML with images test fails

[David Bremner]

Austin Clements <amdragon@MIT.EDU> writes:

> Previously, this would simply indicate that the grep failed without
> any indication of the Emacs output it failed on.  Now we take
> advantage of the test framework's handling of stdout to display the
> incorrect Emacs output if the test fails.

Pushed. Along with a rebuild of emacs on the buildbot, we have all tests
passing on emacs 24.

d