Re: Some testing issues

[Noam Postavsky <npostavs@users.sourceforge.net>]

On Fri, Jul 7, 2017 at 5:22 PM, Stephen Berman <stephen.berman@gmx.net> wrote:
> Several of the tests call the function todo-toggle-view-done-items,
> which contains this code:
>
>           (if (not (pos-visible-in-window-p shown))
>               (recenter)
>
> (`shown' stores the position of the first done item in the category, if
> it is shown.)  When using todo-mode and also when stepping through the
> function with Edebug, it works as expected.

If I comment out the set-window-buffer call it fails for me in Edebug
as well (when running the first time).

By the way, in order to run successfully I did also need the following
patch, otherwise all the tests failed (I think it only happens for me
because I'm running from an Emacs that's living under my home
directory).

modified   test/lisp/calendar/todo-mode-tests.el
@@ -46,6 +46,7 @@ (defmacro with-todo-test (&rest body)
   "Set up an isolated todo-mode test environment."
   (declare (debug (body)))
   `(let* ((todo-test-home (make-temp-file "todo-test-home-" t))
+          (abbreviated-home-dir nil)
           (process-environment (cons (format "HOME=%s" todo-test-home)
                                      process-environment))
           (todo-directory todo-test-data-dir)

>  But when running the tests,
> sometimes -- but not always -- it fails with the error "`recenter'ing a
> window that does not display current-buffer."  In those cases, the test
> failure is prevented by evaluating the following sexp immediately before
> calling todo-toggle-view-done-items:
>
>           (set-window-buffer nil (current-buffer))
>
> When using todo-mode this is not necessary, because the selected window
> always contains the current buffer when the function is called, but
> apparently when running the tests, this isn't always the case, although
> according to Edebug it is, AFAICT.

When I evaluate (selected-window) in edebug before the call, I see
that the selected window is showing the *ert* buffer.

>  And again, the failure only happens
> in some cases, though in those, it is reliably reproducible.  In
> addition, in at least one case, when I rerun the test immediately after
> it fails, then it succeeds.

I think it succeeds the second time because the *ert* buffer is in a
different state.

>  And lastly, in one case, the test succeeds
> without the set-window-buffer call when run interactively, but fails
> without it when run in batch mode from the shell.  Any idea what's going
> on here?  (If anyone wants to take a closer look, comments in the test
> file point out the problematic cases.)

todo-test-toggle-item-header04? I added a `message' call, and it seems
that in batch mode the selected window shows *scratch* whereas in
interactive mode it shows *ert*. I would say the success in
interactive mode is just a coincidence.

>
> A second problem concerns trying to reference a message in a test.  The
> function todo-toggle-view-done-items outputs a message if the category
> contains no done items, and the test captures this by calling
> current-message immediately after todo-toggle-view-done-items.  This
> works when running the test interactively, but in a batch run,
> current-message is nil (though the message is output in the shell).
> Why?

I think it's simply that there is no echo-area or any concept of a
"current" message when running in batch mode.
lread-tests--last-message might be worth looking at:

(defun lread-tests--last-message ()
  (with-current-buffer "*Messages*"
    (save-excursion
      (goto-char (point-max))
      (skip-chars-backward "\n")
      (buffer-substring (line-beginning-position) (point)))))