Heads-up: test failures

Heads-up: test failures

[Nick Dokos]

I just updated to

    Org mode version 9.0.5 (release_9.0.5-260-geb59c7 @ /home/nick/elisp/org-mode/lisp/)

and `make test' gives me:

,----
| Ran 710 tests, 706 results as expected, 4 unexpected (2017-02-13 13:41:50-0500)
| 7 expected failures
| 
| 4 unexpected results:
|    FAILED  test-org-list/move-item-down
|    FAILED  test-org-list/move-item-up
|    FAILED  test-org/custom-properties
|    FAILED  test-org/forward-paragraph
`----

-- 
Nick

Re: Heads-up: test failures

[Marco Wahl]

Nick Dokos <ndokos@gmail.com> writes:

> I just updated to
>
>     Org mode version 9.0.5 (release_9.0.5-260-geb59c7 @ /home/nick/elisp/org-mode/lisp/)
>
> and `make test' gives me:
>
> ,----
> | Ran 710 tests, 706 results as expected, 4 unexpected (2017-02-13 13:41:50-0500)
> | 7 expected failures
> |
> | 4 unexpected results:
> |    FAILED  test-org-list/move-item-down
> |    FAILED  test-org-list/move-item-up
> |    FAILED  test-org/custom-properties
> |    FAILED  test-org/forward-paragraph
> `----

I see these line also.  I thought this was something crazy with my
individual settings.

#+begin_example
Ran 704 tests, 700 results as expected, 4 unexpected (2017-02-13 20:32:03+0100)
8 expected failures

4 unexpected results:
   FAILED  test-org-list/move-item-down
   FAILED  test-org-list/move-item-up
   FAILED  test-org/custom-properties
   FAILED  test-org/forward-paragraph
#+end_example

As Nicolas pointed out these failures don't show in the CI-build at
http://www.randomsample.de:4457/waterfall.

FWIW my environment:

- (version): "GNU Emacs 26.0.50.1 (x86_64-unknown-linux-gnu, GTK+
  Version 3.22.7) of 2017-02-09"

- Actually it's Arch-Linux.

- Recent Org master.


My 2 ct to HTH with this issue

                       Marco

Re: Heads-up: test failures

[Nicolas Goaziou]

Hello,

Marco Wahl <marcowahlsoft@gmail.com> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>> I just updated to
>>
>>     Org mode version 9.0.5 (release_9.0.5-260-geb59c7 @ /home/nick/elisp/org-mode/lisp/)
>>
>> and `make test' gives me:
>>
>> ,----
>> | Ran 710 tests, 706 results as expected, 4 unexpected (2017-02-13 13:41:50-0500)
>> | 7 expected failures
>> |
>> | 4 unexpected results:
>> |    FAILED  test-org-list/move-item-down
>> |    FAILED  test-org-list/move-item-up
>> |    FAILED  test-org/custom-properties
>> |    FAILED  test-org/forward-paragraph
>> `----
>
> I see these line also.  I thought this was something crazy with my
> individual settings.
>
> #+begin_example
> Ran 704 tests, 700 results as expected, 4 unexpected (2017-02-13 20:32:03+0100)
> 8 expected failures
>
> 4 unexpected results:
>    FAILED  test-org-list/move-item-down
>    FAILED  test-org-list/move-item-up
>    FAILED  test-org/custom-properties
>    FAILED  test-org/forward-paragraph
> #+end_example

What is the failure in each case?

Regards,

-- 
Nicolas Goaziou

Re: Heads-up: test failures

[Nick Dokos]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> What is the failure in each case?

The move-down/up failures look like this:

,----
|    passed  512/710  test-org-list/list-navigation
| FOLDED
| FOLDED
| FOLDED
| FOLDED
| Test test-org-list/move-item-down backtrace:
|   (if (unwind-protect (setq value-6667 (apply fn-6665 args-6666)) (set
|   (let (form-description-6669) (if (unwind-protect (setq value-6667 (a
|   (let ((value-6667 (quote ert-form-evaluation-aborted-6668))) (let (f
|   (let ((fn-6665 (function org-invisible-p2)) (args-6666 (list))) (let
|   (progn (org-mode) (let ((point (string-match "<point>" inside-text))
|   (unwind-protect (progn (org-mode) (let ((point (string-match "<point
|   (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
|   (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-b
|   (let ((inside-text (if (stringp "\n- item 1\n  #+BEGIN_CENTER\n  Tex
|   (lambda nil (let ((inside-text (if (stringp "- item 1\n- item 2") "-
|   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
|   ert-run-test([cl-struct-ert-test test-org-list/move-item-down "Test 
|   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
|   ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203G\211\211G\310
|   ert-run-tests-batch("\\(org\\|ob\\)")
|   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
|   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
|   org-test-run-batch-tests("\\(org\\|ob\\)")
|   eval((org-test-run-batch-tests org-test-select-re))
|   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
|   command-line()
|   normal-top-level()
| Test test-org-list/move-item-down condition:
|     (ert-test-failed
|      ((should
|        (org-invisible-p2))
|       :form
|       (org-invisible-p2)
|       :value nil))
|    FAILED  513/710  test-org-list/move-item-down
| FOLDED
| FOLDED
| FOLDED
| FOLDED
| Test test-org-list/move-item-up backtrace:
|   (if (unwind-protect (setq value-6729 (apply fn-6727 args-6728)) (set
|   (let (form-description-6731) (if (unwind-protect (setq value-6729 (a
|   (let ((value-6729 (quote ert-form-evaluation-aborted-6730))) (let (f
|   (let ((fn-6727 (function org-invisible-p2)) (args-6728 (list))) (let
|   (progn (org-mode) (let ((point (string-match "<point>" inside-text))
|   (unwind-protect (progn (org-mode) (let ((point (string-match "<point
|   (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
|   (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-b
|   (let ((inside-text (if (stringp "\n- item 1\n  #+BEGIN_CENTER\n  Tex
|   (lambda nil (let ((inside-text (if (stringp "- item 1\n- item 2") "-
|   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
|   ert-run-test([cl-struct-ert-test test-org-list/move-item-up "Test `o
|   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
|   ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203G\211\211G\310
|   ert-run-tests-batch("\\(org\\|ob\\)")
|   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
|   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
|   org-test-run-batch-tests("\\(org\\|ob\\)")
|   eval((org-test-run-batch-tests org-test-select-re))
|   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
|   command-line()
|   normal-top-level()
| Test test-org-list/move-item-up condition:
|     (ert-test-failed
|      ((should
|        (org-invisible-p2))
|       :form
|       (org-invisible-p2)
|       :value nil))
|    FAILED  514/710  test-org-list/move-item-up
|    passed  515/710  test-org-list/outdent-item
`----

The custom-id failure looks like this:

,----
|    passed  624/710  test-org/custom-id
| Test test-org/custom-properties backtrace:
|   (if (unwind-protect (setq value-10586 (let ((org-custom-properties (
|   (let (form-description-10587) (if (unwind-protect (setq value-10586 
|   (let ((value-10586 (cl-gensym "ert-form-evaluation-aborted-"))) (let
|   (lambda nil (let ((value-10586 (cl-gensym "ert-form-evaluation-abort
|   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
|   ert-run-test([cl-struct-ert-test test-org/custom-properties "Test cu
|   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
|   ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203G\211\211G\310
|   ert-run-tests-batch("\\(org\\|ob\\)")
|   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
|   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
|   org-test-run-batch-tests("\\(org\\|ob\\)")
|   eval((org-test-run-batch-tests org-test-select-re))
|   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
|   command-line()
|   normal-top-level()
| Test test-org/custom-properties condition:
|     (ert-test-failed
|      ((should
|        (let
| 	   (...)
| 	 (org-test-with-temp-text "* H
| :PROPERTIES:
| <point>:FOO: val
| :END:
| " ... ...)))
|       :form
|       (let
| 	  ((org-custom-properties ...))
| 	(org-test-with-temp-text "* H
| :PROPERTIES:
| <point>:FOO: val
| :END:
| "
| 	  (org-toggle-custom-properties-visibility)
| 	  (org-invisible-p2)))
|       :value nil))
|    FAILED  625/710  test-org/custom-properties
`----

The forward-paragraph failrue looks like this:

,----
|    passed  645/710  test-org/forward-element
| Test test-org/forward-paragraph backtrace:
|   (if (unwind-protect (setq value-10777 (let ((inside-text (if (string
|   (let (form-description-10778) (if (unwind-protect (setq value-10777 
|   (let ((value-10777 (cl-gensym "ert-form-evaluation-aborted-"))) (let
|   (lambda nil (let ((value-10769 (cl-gensym "ert-form-evaluation-abort
|   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
|   ert-run-test([cl-struct-ert-test test-org/forward-paragraph "Test `o
|   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
|   ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203G\211\211G\310
|   ert-run-tests-batch("\\(org\\|ob\\)")
|   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
|   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
|   org-test-run-batch-tests("\\(org\\|ob\\)")
|   eval((org-test-run-batch-tests org-test-select-re))
|   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
|   command-line()
|   normal-top-level()
| Test test-org/forward-paragraph condition:
|     (ert-test-failed
|      ((should
|        (org-test-with-temp-text "#+BEGIN_CENTER
| P1
| 
| P2
| #+END_CENTER
| P3"
| 	 (org-hide-block-toggle)
| 	 (org-forward-paragraph)
| 	 (looking-at "P3")))
|       :form
|       (let
| 	  ((inside-text ...)
| 	   (org-mode-hook nil))
| 	(with-temp-buffer
| 	  (org-mode)
| 	  (let ... ...)
| 	  (org-hide-block-toggle)
| 	  (org-forward-paragraph)
| 	  (looking-at "P3")))
|       :value nil))
|    FAILED  646/710  test-org/forward-paragraph
`----

Version info:
GNU Emacs 26.0.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 3.20.9) of 2017-02-07
Org mode version 9.0.5 (release_9.0.5-260-geb59c7 @ /home/nick/elisp/org-mode/lisp/)

Note: I don't remember the last time that I ran `make test', so these may (or may not) be
recent.
And another note: I deleted some NULs from the backtraces to keep Gnus from asking.

-- 
Nick

Re: Heads-up: test failures

[Colin Baxter]

On Mon, Feb 13 2017, Nick Dokos wrote:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> What is the failure in each case?
>
---- snip ----

Ran 711 tests, 711 results as expected (2017-02-14 07:58:32+0000)
8 expected failures

org-mode release_9.0.5-282-g2e32709
emacs-25.1.1; 3.2.0-4-686-pae #1 SMP Debian 3.2.84-1 i686 GNU/Linux

Re: Heads-up: test failures

[Nick Dokos]

Colin Baxter <m43cap@yandex.com> writes:

> On Mon, Feb 13 2017, Nick Dokos wrote:
>
>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>
>>> What is the failure in each case?
>>
> ---- snip ----
>
> Ran 711 tests, 711 results as expected (2017-02-14 07:58:32+0000)
> 8 expected failures
>
> org-mode release_9.0.5-282-g2e32709
> emacs-25.1.1; 3.2.0-4-686-pae #1 SMP Debian 3.2.84-1 i686 GNU/Linux
>
>

It might be that emacs 26 shows the failures (both Marco Wahl and I are
running that, but Colin Baxter runs emacs 25 and does not see them).

I simulated the move-down test by hand:

,----
|   ;; Preserve children visibility.
|   (org-test-with-temp-text "* Headline
| - item 1
|   - sub-item 1
|     sub-body 1
| - item 2
|   - sub-item 2
|     sub-body 2"
|     (let ((org-cycle-include-plain-lists t))
|       (search-forward "- sub-item 1")
|       (org-cycle)
|       (search-forward "- sub-item 2")
|       (org-cycle))
|     (search-backward "- item 1")
|     (org-move-item-down)
|     (search-forward "sub-body 1")
|     (should (org-invisible-p2))
|     (search-backward "sub-body 2")
|     (should (org-invisible-p2)))
`----

The   (search-forward "sub-body 1") ends up in a place where (org-invisible-p2) returns t,
but the (search-backward "sub-body 2") end up in a place where it returns nil (C-u C-x = shows
that it's on the RET, at the end of the sub-item 1 line).

Maybe something about such text properties got changed in recent emacs?

-- 
Nick

Re: Heads-up: test failures

[Colin Baxter]

On Tue, Feb 14 2017, Nick Dokos wrote:

> Colin Baxter <m43cap@yandex.com> writes:
>
>> On Mon, Feb 13 2017, Nick Dokos wrote:
>>
>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>>
>>>> What is the failure in each case?
>>>
>> ---- snip ----
>>
>> Ran 711 tests, 711 results as expected (2017-02-14 07:58:32+0000)
>> 8 expected failures
>>
>> org-mode release_9.0.5-282-g2e32709
>> emacs-25.1.1; 3.2.0-4-686-pae #1 SMP Debian 3.2.84-1 i686 GNU/Linux
>>
>>
>
> It might be that emacs 26 shows the failures (both Marco Wahl and I are
> running that, but Colin Baxter runs emacs 25 and does not see them).


Unfortunately, the tests pass using emacs-26 too:

Ran 711 tests, 711 results as expected (2017-02-14 21:21:38+0000)
8 expected failures

org-version: 9.0.5 (release_9.0.5-288-g4caad0)
GNU Emacs 26.0.50.1 (i686-pc-linux-gnu, GTK+ Version 3.4.2) of 2017-02-11


Best wishes,

Colin.

Re: Heads-up: test failures

[Nick Dokos]

Colin Baxter <m43cap@yandex.com> writes:

> On Tue, Feb 14 2017, Nick Dokos wrote:
>
>> Colin Baxter <m43cap@yandex.com> writes:
>>
>>> On Mon, Feb 13 2017, Nick Dokos wrote:
>>>
>>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>>>
>>>>> What is the failure in each case?
>>>>
>>> ---- snip ----
>>>
>>> Ran 711 tests, 711 results as expected (2017-02-14 07:58:32+0000)
>>> 8 expected failures
>>>
>>> org-mode release_9.0.5-282-g2e32709
>>> emacs-25.1.1; 3.2.0-4-686-pae #1 SMP Debian 3.2.84-1 i686 GNU/Linux
>>>
>>>
>>
>> It might be that emacs 26 shows the failures (both Marco Wahl and I are
>> running that, but Colin Baxter runs emacs 25 and does not see them).
>
>
> Unfortunately, the tests pass using emacs-26 too:
>
> Ran 711 tests, 711 results as expected (2017-02-14 21:21:38+0000)
> 8 expected failures
>
> org-version: 9.0.5 (release_9.0.5-288-g4caad0)
> GNU Emacs 26.0.50.1 (i686-pc-linux-gnu, GTK+ Version 3.4.2) of 2017-02-11
>
>

Well, I'm running 26.0.50.2, so there is still a possible out. But Marco Wahl
is running 26.0.50.1, so there goes that theory :-)

I usually run a shell within emacs and I ran `make test' from there earlier.
I tried doing a `make test' from a shell in an `emacs -q' and also just from
a plain xterm: I still get the four failures.

-- 
Nick

Re: Heads-up: test failures

[John Hendy]

On Mon, Feb 13, 2017 at 12:48 PM, Nick Dokos <ndokos@gmail.com> wrote:
> I just updated to
>
>     Org mode version 9.0.5 (release_9.0.5-260-geb59c7 @ /home/nick/elisp/org-mode/lisp/)
>
> and `make test' gives me:

Since others are adding in to the party! I ran after a fresh pull on
my normal setup.

$ uname -a
Linux whammy 4.9.8-1-ARCH #1 SMP PREEMPT Mon Feb 6 12:59:40 CET 2017
x86_64 GNU/Linux

$ emacs --version
GNU Emacs 25.1.1

M-x org-version
Org mode version 9.0.5 (release_9.0.5-288-g4caad0 @
/home/jwhendy/.elisp/org/lisp/)

Ran 705 tests, 705 results as expected (2017-02-14 20:12:26-0600)
9 expected failures

Since there was some mention of the emacs version, I just grabbed the
Arch AUR emacs-git package and re-ran. This is with 26.0.50.1:

Ran 705 tests, 701 results as expected, 4 unexpected (2017-02-14 20:28:42-0600)
9 expected failures

4 unexpected results:
   FAILED  test-org-list/move-item-down
   FAILED  test-org-list/move-item-up
   FAILED  test-org/custom-properties
   FAILED  test-org/forward-paragraph


Looks like emacs version is involved.


John


>
> --
> Nick
>
>
>

Re: Heads-up: test failures

[Vladimir Lomov]

Hello,
** John Hendy [2017-02-14 20:32:59 -0600]:

> On Mon, Feb 13, 2017 at 12:48 PM, Nick Dokos <ndokos@gmail.com> wrote:
>> I just updated to
>>
>>     Org mode version 9.0.5 (release_9.0.5-260-geb59c7 @ /home/nick/elisp/org-mode/lisp/)
>>
>> and `make test' gives me:
> 
> Since others are adding in to the party! I ran after a fresh pull on
> my normal setup.
> 
> $ uname -a
> Linux whammy 4.9.8-1-ARCH #1 SMP PREEMPT Mon Feb 6 12:59:40 CET 2017
> x86_64 GNU/Linux
> 
> $ emacs --version
> GNU Emacs 25.1.1
> 
> M-x org-version
> Org mode version 9.0.5 (release_9.0.5-288-g4caad0 @
> /home/jwhendy/.elisp/org/lisp/)
> 
> Ran 705 tests, 705 results as expected (2017-02-14 20:12:26-0600)
> 9 expected failures
> 
> Since there was some mention of the emacs version, I just grabbed the
> Arch AUR emacs-git package and re-ran. This is with 26.0.50.1:
> 
> Ran 705 tests, 701 results as expected, 4 unexpected (2017-02-14 20:28:42-0600)
> 9 expected failures
> 
> 4 unexpected results:
>    FAILED  test-org-list/move-item-down
>    FAILED  test-org-list/move-item-up
>    FAILED  test-org/custom-properties
>    FAILED  test-org/forward-paragraph
> 
> 
> Looks like emacs version is involved.

I want to join this party:
-------------- 8< -------------
$ uname -a
Linux smoon4.vl-lomov.ru 4.9.9-1-ck-sandybridge #1 SMP PREEMPT Thu Feb 9 16:04:12 EST 2017 x86_64 GNU/Linux
-------------- 8< -------------

-------------- 8< -------------
$ cat /etc/os-release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
ID_LIKE=archlinux
ANSI_COLOR="0;36"
HOME_URL="https://www.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
-------------- 8< -------------

-------------- 8< -------------
$ pacman -Qs emacs-git | head -n 1
local/emacs-git 25.2_rc1_2988_g31b4d9a137-1
-------------- 8< -------------

-------------- 8< -------------
$ emacs --version
GNU Emacs 26.0.50.1
Copyright (C) 2017 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.
-------------- 8< -------------

Cloned git repo to ~/tmp/emacs-org-mode and run in it
-------------- 8< -------------
$ make compile
-------------- 8< -------------

-------------- 8< -------------
$ LANG=C make test

Ran 705 tests, 699 results as expected, 6 unexpected (2017-02-15 11:43:22+0800)
9 expected failures

6 unexpected results:
   FAILED  test-org-clock/clocktable/scope
   FAILED  test-org-colview/columns-summary
   FAILED  test-org-list/move-item-down
   FAILED  test-org-list/move-item-up
   FAILED  test-org/custom-properties
   FAILED  test-org/forward-paragraph

make: *** [mk/targets.mk:104: test] Error 1
-------------- 8< -------------

Not sure what this mean, but if I run the 'make test' with my default
locale (ru_RU.UTF-8) then it reported about 10 unexpected results:

-------------- 8< -------------
Ran 705 tests, 695 results as expected, 10 unexpected (2017-02-15 11:47:39+0800)
9 expected failures

10 unexpected results:
   FAILED  test-org-clock/clocktable/scope
   FAILED  test-org-colview/columns-summary
   FAILED  test-org-list/move-item-down
   FAILED  test-org-list/move-item-up
   FAILED  test-org/add-planning-info
   FAILED  test-org/clone-with-time-shift
   FAILED  test-org/custom-properties
   FAILED  test-org/deadline
   FAILED  test-org/forward-paragraph
   FAILED  test-org/schedule

make: *** [mk/targets.mk:104: test] Ошибка 1
-------------- 8< -------------

I will try to repeat the test with other emacs commit (current one is
31b4d9a137).

-- 
Welcome to boggle - do you want instructions?

D    G    G    O

O    Y    A    N

A    D    B    T

K    I    S    P
Enter words:
>

Re: Heads-up: test failures

[Colin Baxter]

Hi Nick


On Tue, Feb 14 2017, Nick Dokos wrote:

> Colin Baxter <m43cap@yandex.com> writes:
>
>> On Tue, Feb 14 2017, Nick Dokos wrote:
>>
>>> Colin Baxter <m43cap@yandex.com> writes:
>>>
>>>> On Mon, Feb 13 2017, Nick Dokos wrote:
>>>>
>>>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>>>>
>>>>>> What is the failure in each case?
>>>>>
>>>> ---- snip ----
>>>>
>>>> Ran 711 tests, 711 results as expected (2017-02-14 07:58:32+0000)
>>>> 8 expected failures
>>>>
>>>> org-mode release_9.0.5-282-g2e32709
>>>> emacs-25.1.1; 3.2.0-4-686-pae #1 SMP Debian 3.2.84-1 i686 GNU/Linux
>>>>
>>>>
>>>
>>> It might be that emacs 26 shows the failures (both Marco Wahl and I are
>>> running that, but Colin Baxter runs emacs 25 and does not see them).
>>
>>
>> Unfortunately, the tests pass using emacs-26 too:
>>
>> Ran 711 tests, 711 results as expected (2017-02-14 21:21:38+0000)
>> 8 expected failures
>>
>> org-version: 9.0.5 (release_9.0.5-288-g4caad0)
>> GNU Emacs 26.0.50.1 (i686-pc-linux-gnu, GTK+ Version 3.4.2) of 2017-02-11
>>
>>
>
> Well, I'm running 26.0.50.2, so there is still a possible out. But Marco Wahl
> is running 26.0.50.1, so there goes that theory :-)
>
> I usually run a shell within emacs and I ran `make test' from there earlier.
> I tried doing a `make test' from a shell in an `emacs -q' and also just from
> a plain xterm: I still get the four failures.


Sorry, my mistake. Because of my setup I was not using emacs-26 to make
the test but inadvertently emacs-25.

Using emacs-26, I can now confirm:

Ran 711 tests, 707 results as expected, 4 unexpected (2017-02-15 05:39:31+0000)
8 expected failures

4 unexpected results:
   FAILED  test-org-list/move-item-down
   FAILED  test-org-list/move-item-up
   FAILED  test-org/custom-properties
   FAILED  test-org/forward-paragraph

Sorry about that.

Best wishes,

Colin.