Hi!

Unfortunately, I can't comment on the actual error found, as I wasn't
involved in the fix...

Anyway, I am positively glad that you have tests for font-lock rules.
However, using htmlfontify for this doesn't seem like the best tool for
this. 1) The output of htmlfontify may change in the future, in which case
you will need to rewrite all your tests, 2) HTML is hard to read, and 3)
The ert explainer doesn't really help you here. If example, in this case
all it tells you is that the resulting html strings have different lengths.
On other words, it may work for snippets like this, but it would not scale
well.

A couple of years ago I wrote a tool, faceup (https://github.com/
Lindydancer/faceup), specifically for testing highlighting in Emacs using
ert. It uses a simple markup language, so the result is easier to read than
when using HTML. The reference text can be used inline (like in your test)
or stored in a corresponding ".faceup" file, which makes it possible to
test files thousands of lines long. When a test fails, the ert explainer
will tell you which lines contained errors, what that lines contained, and
what they should contain, so it's much easier to find errors. I have
successfully used this is several of my font-lock packages. In addition,
I've accumulated a number of source files in various programming languages
in a regression test suite for font-lock (https://github.com/
Lindydancer/font-lock-regression-suite).

If you would like to modernise your font-lock test, I would be happy to
contribute it to Emacs (as I suggested in bug#16063).

    -- Anders

On Fri, Sep 22, 2017 at 7:01 PM, Glenn Morris <rgm@gnu.org> wrote:

>
> Running 5 tests (2017-09-22 09:57:54-0700)
>    passed  1/5  bat-test-fill-paragraph-comment
> hfy batch mode ( *temp*-973592:bat-mode)
> Test bat-test-fontification-iter-var-1 backtrace:
>   signal(ert-test-failed (((should (equal (bat-test-fontify "echo %%a\
>   ert-fail(((should (equal (bat-test-fontify "echo %%a\necho %%~dp1\ne
>   (if (unwind-protect (setq value-17 (apply fn-15 args-16)) (setq form
>   (let (form-description-19) (if (unwind-protect (setq value-17 (apply
>   (let ((value-17 'ert-form-evaluation-aborted-18)) (let (form-descrip
>   (let* ((fn-15 (function equal)) (args-16 (condition-case err (let ((
>   (closure (t) nil (let* ((fn-15 (function equal)) (args-16 (condition
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name bat-test-fontification-iter-var-1 :do
>   ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test
>   ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
>   ert-run-tests-batch(nil)
>   ert-run-tests-batch-and-exit(nil)
>   eval((ert-run-tests-batch-and-exit nil))
>   command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/progmodes/bat-mode-t
>   command-line()
>   normal-top-level()
> Test bat-test-fontification-iter-var-1 condition:
>     (ert-test-failed
>      ((should
>        (equal
>         (bat-test-fontify "echo %%a
> echo %%~dp1
> echo %%~$PATH:I")
>         "<span class=\"builtin\">echo</span> %%<span
> class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</
> span>
> <span class=\"builtin\">echo</span> %%~$<span
> class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</
> span>"))
>       :form
>       (equal
>        #("<span class=\"builtin\">echo</span> %%<span
> class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</
> span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>"
> 22 26
> (face font-lock-builtin-face)
> 64 65
> (face font-lock-variable-name-face)
> 95 99
> (face font-lock-builtin-face)
> 137 141
> (face font-lock-variable-name-face)
> 171 175
> (face font-lock-builtin-face)
> 213 221
> (face font-lock-variable-name-face))
>        "<span class=\"builtin\">echo</span> %%<span
> class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</
> span>
> <span class=\"builtin\">echo</span> %%~$<span
> class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</
> span>")
>       :value nil :explanation
>       (arrays-of-different-length 228 263
>                                   #("<span class=\"builtin\">echo</span>
> %%<span class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</
> span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>"
> 22 26
> (face font-lock-builtin-face)
> 64 65
> (face font-lock-variable-name-face)
> 95 99
> (face font-lock-builtin-face)
> 137 141
> (face font-lock-variable-name-face)
> 171 175
> (face font-lock-builtin-face)
> 213 221
> (face font-lock-variable-name-face))
>                                   "<span class=\"builtin\">echo</span>
> %%<span class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</
> span>
> <span class=\"builtin\">echo</span> %%~$<span
> class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>"
> first-mismatch-at 109)))
>    FAILED  2/5  bat-test-fontification-iter-var-1
> hfy batch mode ( *temp*-454955:bat-mode)
>    passed  3/5  bat-test-fontification-var-decl
> hfy batch mode ( *temp*-493544:bat-mode)
>    passed  4/5  bat-test-fontification-var-delayed-exp
> hfy batch mode ( *temp*-832943:bat-mode)
>    passed  5/5  bat-test-fontification-var-exp
>
> Ran 5 tests, 4 results as expected, 1 unexpected (2017-09-22 09:57:54-0700)
>
> 1 unexpected results:
>    FAILED  bat-test-fontification-iter-var-1
>
>