Re: Display test suite

[Eli Zaretskii <eliz@gnu.org>]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Thu, 7 Jul 2016 11:47:54 -0400
> 
> Richard recently suggested giving Emacs the ability to take a screenshot of itself. Could we imagine using this to run regression tests of the display engine?

Maybe I'm missing something, but I don't see how.  A test suite needs
to include the expected results for each test.  Even if you include an
image file for each test that shows what should be displayed, AFAIU
the image will be specific to the display resolution, number of
colors, dimensions, etc., so inappropriate for automatic comparison
with the screenshot on the target machine.  Am I right?

Also, comparing images that could have a small number of pixels
different in some corner is error-prone.

IMO, a test suite for the display engine needs a way to describe
precisely what's on the screen, in some language, and then compare
those descriptions.

> Of course, it could be hard to get screeenshots to looks exactly the same for everyone. Instead, we could have a makefile target that builds a large collection of these screenshots, and another one that rebuilds it and compares against a previously generated one.

But then each time I start using a new machine, my reference point is
reset to a newer date, and I lose the ability for comparing against
older versions.

Thanks.