Re: burden of maintainance

[Tassilo Horn <tsdh@gnu.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> > We are mis-communicating again: I meant features which display
>> > something or involve interactive keyboard input.
>> 
>> Emacs is in a better position here than most other tools because it
>> has keyboard macros and can easily inspect the things it displays,
>> e.g., the text properties or overlays at point.
>
> We may be mis-communicating here.  Because I cannot see how examining
> text properties or overlays can test whether their display on screen
> is correct.  What am I missing?  Can you show an example of what you
> had in mind?

No, if something is actually displayed correctly cannot be tested like
so.  What I had in mind were things like testing if fontification works
properly by checking if the 'face property is set to some expected face
when point is one some keyword.

> If we are talking about testing the display engine, it should be
> possible and relatively easy to provide Lisp access to the "glyph
> matrices", which are data structures the display engine builds that
> describe what _should_ be on the screen.  You can see one specialized
> example of that in the function bidi-resolved-levels which I wrote for
> testing the bidirectional display.
>
> But doing the above is only half the way.  The other half is the
> terminal-specific back-end, either X, w32, NS, or TTY, which actually
> converts the glyph matrices to draw commands and delivers the results
> to the glass.  To test that, we need some way of "sniffing" the
> results of this drawing, which requires terminal-specific
> infrastructure.
>
> We then need some "language" to describe the expected results to be
> displayed on the screen.  For TTY, this is almost trivial, but for
> sophisticated GUI display features it's more complicated (think of
> fonts, stretches of white space, composed characters, fringe bitmaps,
> tool-bar icons, tooltips, etc.).  We also need a way of describing the
> results of cursor movement.
>
> When all of these are implemented, then yes, writing display tests
> should be relatively easy.  But today all this infrastructure doesn't
> exist.  We don't even have its detailed design.

That's obviously not what I've head in mind.  But I don't think that's
an area where we need a special focus on tests.  At least I have the
feeling that there are only few bugs wrt. the display engine.

>> I can think of something like "ERT keyboard-macro tests" where you
>> would basically record a keyboard macro and during that, you have
>> some way to add some assertions before resuming recording.  The whole
>> recorded thing could then be stored as an ERT test.
>
> For testing keyboard input, we need more than that.  For example,
> consider the simple feature of input history -- you'd need a way to
> inject M-p, M-n, etc., and a way of recording the text that is
> displayed as result and returned to the caller.
>
> And please don't forget that keyboard input is only one kind of events
> supported by Emacs.  There are others, starting with mouse.  We'd need
> ways to simulate or trigger all of them.

Yes, eventually.  But it would at least be a first step.  And this
hypothetical recorder could also be exposed to the user for creating
replayable recipes for bugs they report.  Where a tester would add
assertions, the user would just write what he's expecting, and whoever
fixes the bug could easily turn the string into assertions, and voila,
there's the regression test for the bug.

Bye,
Tassilo