Autoconf autotest compilation-error-regexp

Autoconf autotest compilation-error-regexp

[Nikolai Weibull]

Hi!

Has anyone written any autoconf autotest compilation-error-regexps?  I
can’t seem to get it to work.

The output, when running the testsuite with -v is rather poor and
looks like this (where all capitals replace generic information):

1. testsuite.at:5: testing TEST TITLE ...
./testsuite.at:11: COMMAND
1. testsuite.at:5:  ok

2. testsuite.at:16: testing TEST TITLE ...
./testsuite.at:22: COMMAND
--- -   2012-11-18 13:00:01.000000000 +0100
+++ .../stdout       2012-11-18 13:00:01.000000000 +0100
DIFF OUTPUT

2. testsuite.at:16:  FAILED (testsuite.at:22)

My attempt was

(add-to-list 'compilation-error-regexp-alist-alist
                  '(autotest-footer
                    "^[1-9][0-9]*\\. \\([^\n :]+\\):\\([1-9][0-9]*\\):
 \\(?:ok\\|FAILED\\)"
                    1 2 nil 0 1))
     (add-to-list 'compilation-error-regexp-alist 'autotest-footer)
     (add-to-list 'compilation-error-regexp-alist-alist
                  '(autotest-error
                    "^[1-9][0-9]*\\. [^\n :]+:[1-9][0-9]*: testing
[^\n]*\n\
\\([^\n :]+\\):\\([1-9][0-9]*\\): [^\n]+\n\\(?:\\([^-]\\)\\|---\\)"
                    1 2 nil (3 . 3) 1))
     (add-to-list 'compilation-error-regexp-alist 'autotest-error)

but that doesn’t seem to work.  It matches the ok/failed part OK, but
the other doesn’t want to stick.

Re: Autoconf autotest compilation-error-regexp

[Nikolai Weibull]

On Sun, Nov 18, 2012 at 1:06 PM, Nikolai Weibull <now@bitwi.se> wrote:

> Has anyone written any autoconf autotest compilation-error-regexps?  I
> can’t seem to get it to work.

I managed to get some time to put something together that works OK:

     (add-to-list 'compilation-error-regexp-alist-alist
                  '(autotest-header
                    "^\\([1-9][0-9]*\\. \\([^\n
:]+\\.at\\):\\([1-9][0-9]*\\)\\): \\(?:
FAILED\\|WARNIN\\(G\\)\\|\\(testing\\| ok\\)\\)"
                    2 3 nil (4 . 5) 1))
     (add-to-list 'compilation-error-regexp-alist 'autotest-header)
     (add-to-list 'compilation-error-regexp-alist-alist
                  '(autotest-check-error
                    "^\\(\\([^\n :]+\\.at\\):\\([1-9][0-9]*\\)\\):
[^\n]+\n\\(?:\\([^-]\\)\\|--- \\)"
                    2 3 nil (nil . 4) 1))
     (add-to-list 'compilation-error-regexp-alist 'autotest-check-error)

I’m sure it can be improved, but this currently seems to work fine for my needs.

Autoconf autotest compilation-error-regexp

[Nikolai Weibull]

Hi!

I’ve long been trying to set up a working autoconf autotest
compilation-error-regexp.  Autoconf’s autotest has some rather funky
output that’s quite difficult to parse correctly.  In fact, it only
outputs usable paths if given the -v command-line option, in which
case the output is (where all capitals replace generic information):

1. testsuite.at:5: testing TEST TITLE ...
./testsuite.at:11: COMMAND
1. testsuite.at:5:  ok

2. testsuite.at:16: testing TEST TITLE ...
./testsuite.at:22: COMMAND
--- -   2012-11-18 13:00:01.000000000 +0100
+++ .../stdout       2012-11-18 13:00:01.000000000 +0100
DIFF OUTPUT

2. testsuite.at:16:  FAILED (testsuite.at:22)

All lines prefixed by “ORDINAL. ” are headers.  The path is always
displayed without any directory part, so this information is next to
useless for a compilation-error-regexp.  The problem is that to the
gnu compilation-error-regexp, such lines look like actual errors,
which they are not.  We therefore need a compilation-error-regexp that
makes sure that these lines are ignored.

The first line after these header lines displays the command being run
to execute the test.  If this line is followed by a diff, the test
failed.

Given this specification, this is what I’ve come up with:

(add-to-list 'compilation-error-regexp-alist-alist
             '(autotest-header
               "^\\([1-9][0-9]*\\. \\([^\n
:]+\\.at\\):\\([1-9][0-9]*\\)\\): \\(?:
FAILED\\|WARNIN\\(G\\)\\|\\(testing\\| ok\\)\\)"
               2 3 nil (4 . 5) 1))
(add-to-list 'compilation-error-regexp-alist 'autotest-header)

(add-to-list 'compilation-error-regexp-alist-alist
             '(autotest-check
               "^\\(\\([^\n :]+\\.at\\):\\([1-9][0-9]*\\)\\): "
               2 3 nil 0 1))
(add-to-list 'compilation-error-regexp-alist 'autotest-check)

(add-to-list 'compilation-error-regexp-alist-alist
             '(autotest-check-error
               "^\\(\\([^\n :]+\\.at\\):\\([1-9][0-9]*\\)\\): [^\n]+\n--- "
               2 3 nil 2 1))
(add-to-list 'compilation-error-regexp-alist 'autotest-check-error)

The first issue is that there’s no way to tell compile.el that a
compilation-error-regexp should simply be ignored.  My suggested
solution is to allow HYPERLINK to be given as -1 (or 3) and for this
to mean that the match should be ignored.

The second issue is that it seems that multi-line regexps might miss
when there’s another regexp that can match on a single line
(autotest-check-error and autotest-check).  This happens
intermittently, and I suppose it’s due to how this is being handled,
where a scan gives a match on the autotest-check regexp if not enough
output has been produced yet to be matched by the autotest-check-error
regexp.  (Autotest-check is needed so that the line won’t be matched
by the gnu regexp.)  I don’t know how to get around this, beyond
applying patches to Autoconf that make the output a bit more sane.

Anyway, any suggestions on how to proceed?

Are there better ways of handling this?

Does it seem reasonable to add support for ignoring a matched line?

Is there any way of avoiding the intermittent issues with regexps that
match depending on how much output has been produced?