bug#33872: flymake-tests different-diagnostic-types test fails

bug#33872: flymake-tests different-diagnostic-types test fails

[Michał Nazarewicz]

The flymake-tests different-diagnostic-types test fails on HEAD
(commit 9fe788a1fa02c6b717c709773f3cca7bc8b2ebe6) with gcc 8.2.0
(though I’m not sure if gcc version is relevant) as can be seen in
test log:

---- >8 ----------------------------------------------------------------
Running 9 tests (2018-12-25 13:58:53+0000, selector `(not (or (tag
:expensive-test) (tag :unstable)))')
Test different-diagnostic-types backtrace:
  signal(ert-test-failed (((should (eq 'flymake-error (face-at-point))
  ert-fail(((should (eq 'flymake-error (face-at-point))) :form (eq fly
  #f(compiled-function () #<bytecode 0x4807dd>)()
  flymake-tests--call-with-fixture(#f(compiled-function () #<bytecode
  #f(compiled-function () #<bytecode 0x4807e9>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name different-diagnostic-types :documenta
  ert-run-or-rerun-test(#s(ert--stats :selector (not (or ... ...)) :te
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/progmodes/flymake-te
  command-line()
  normal-top-level()
Test different-diagnostic-types condition:
    (ert-test-failed
     ((should
       (eq 'flymake-error
        (face-at-point)))
      :form
      (eq flymake-error flymake-note)
      :value nil))
   FAILED  1/9  different-diagnostic-types (1.058400 sec)
   passed  2/9  dummy-backends (1.003778 sec)
   passed  3/9  eob-region-and-trailing-newline (0.000181 sec)
   passed  4/9  included-c-header-files (2.029273 sec)
   passed  5/9  perl-backend (1.024491 sec)
   passed  6/9  recurrent-backend (1.006058 sec)
  skipped  7/9  ruby-backend (0.000575 sec)
   passed  8/9  warning-predicate-function-gcc (1.015344 sec)
   passed  9/9  warning-predicate-rx-gcc (1.017081 sec)

Ran 9 tests, 7 results as expected, 1 unexpected, 1 skipped
(2018-12-25 13:59:01+0000, 8.225831 sec)

1 unexpected results:
   FAILED  different-diagnostic-types

1 skipped results:
  SKIPPED  ruby-backend
---- 8< ----------------------------------------------------------------

The issue is that flymake does not recognise

    #include "some-problems.h"

line in the test file as problematic even though gcc complains about
the header file:

---- >8 ----------------------------------------------------------------
$ make ./test/lisp/progmodes/flymake-resources/errors-and-warnings
cc     test/lisp/progmodes/flymake-resources/errors-and-warnings.c
-o test/lisp/progmodes/flymake-resources/errors-and-warnings
In file included from
test/lisp/progmodes/flymake-resources/errors-and-warnings.c:3:
test/lisp/progmodes/flymake-resources/some-problems.h:3:1: warning:
data definition has no type or storage class
 strange;
 ^~~~~~~
test/lisp/progmodes/flymake-resources/some-problems.h:3:1: warning:
type defaults to ‘int’ in declaration of ‘strange’ [-Wimplicit-int]
test/lisp/progmodes/flymake-resources/some-problems.h:5:1: error:
unknown type name ‘sint’; did you mean ‘int’?
 sint main();
 ^~~~
 int
test/lisp/progmodes/flymake-resources/errors-and-warnings.c: In function ‘main’:
test/lisp/progmodes/flymake-resources/errors-and-warnings.c:9:12:
warning: overflow in conversion from ‘int’ to ‘char’ changes value
from ‘1000’ to ‘-24’ [-Woverflow]
   char c = 1000; /* a note and a warning */
            ^~~~
test/lisp/progmodes/flymake-resources/errors-and-warnings.c:11:8:
error: redeclaration of ‘c’ with no linkage
   char c; if (bla == (void*)3); /* an error, and two warnings */
        ^
test/lisp/progmodes/flymake-resources/errors-and-warnings.c:9:8: note:
previous definition of ‘c’ was here
   char c = 1000; /* a note and a warning */
        ^
test/lisp/progmodes/flymake-resources/errors-and-warnings.c:11:19:
warning: comparison between pointer and integer
   char c; if (bla == (void*)3); /* an error, and two warnings */
                   ^~
make: *** [<builtin>:
test/lisp/progmodes/flymake-resources/errors-and-warnings] Error 1
---- 8< ----------------------------------------------------------------

Commenting the first check in the test (as shown below) makes it pass:

---- >8 ----------------------------------------------------------------
diff --git a/test/lisp/progmodes/flymake-tests.el
b/test/lisp/progmodes/flymake-tests.el
index ef1fd19e86..f16db1e09e 100644
--- a/test/lisp/progmodes/flymake-tests.el
+++ b/test/lisp/progmodes/flymake-tests.el
@@ -151,8 +151,8 @@ ruby-mode-hook
     (flymake-tests--with-flymake
         ("errors-and-warnings.c")
       (flymake-goto-next-error)
-      (should (eq 'flymake-error (face-at-point)))
-      (flymake-goto-next-error)
+      ;; (should (eq 'flymake-error (face-at-point)))
+      ;; (flymake-goto-next-error)
       (should (eq 'flymake-note (face-at-point)))
       (flymake-goto-next-error)
       (should (eq 'flymake-warning (face-at-point)))
---- >8 ----------------------------------------------------------------

-- 
Best regards
ミハウ “𝓶𝓲𝓷𝓪86” ナザレヴイツ
«If at first you don’t succeed, give up skydiving»

bug#33872: flymake-tests different-diagnostic-types test fails

[João Távora]

Michał Nazarewicz <mina86@mina86.com> writes:

> The flymake-tests different-diagnostic-types test fails on HEAD
> (commit 9fe788a1fa02c6b717c709773f3cca7bc8b2ebe6) with gcc 8.2.0
> (though I’m not sure if gcc version is relevant) as can be seen in
> test log:

Thanks for the report.  I reproduced and fixed the bug in
04d3315271cd1357fbbc192505d00c760be02952.  GCC 8.2.0 doesn't report the
column in "In file included from" errors.

João