[ANN] We are now regularily testing Org main branch against Emacs 26, 27, 28

[ANN] We are now regularily testing Org main branch against Emacs 26, 27, 28

[Bastien]

Hi all,

thanks to joint efforts by Christian Köstlin and Ihor, we are running
the Org test suite from the Org main branch against Emacs 26, 27, 28.

Every three hours, a script from the orgmode.org server checks if the
Org main branch changed.

If yes, we run the Org test suite against Emacs 26, 27 and 28.

If a test fails, a failure report is sent to a new mailing list:
https://lists.sr.ht/~bzg/org-build-failures and we can inspect what
did go wrong.

If a test fails again (i.e. there was a failure, then the repo was
updated, then there is a failure again for the same test), reports are
sent to Ihor, Christian and me, not to the dedicated list.

Tests are performed by running builds on sr.ht.  You can read the
builds in this new repository: https://git.sr.ht/~bzg/org-mode-tests

We are now testing this setup: if it useful enough, we will redirect
failure reports to the Org mailing list, while using the new one for
repeated failures, so as to not spam the list.

In the future, resources at https://git.sr.ht/~bzg/org-mode-tests
will also perhaps help contributors defining new ways on how to run
tests, beyond the current way ("make test").

Patch submitters are required to run the tests themselves before
submitting a patch: we don't rely on this new setup to test patches,
only to catch errors that may inadvertently slip through the cracks.

Thanks again to Christian and Ihor for setting this up!

-- 
 Bastien

Re: [ANN] We are now regularily testing Org main branch against Emacs 26, 27, 28

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

On Sun, Oct 16, 2022 at 2:04 AM Bastien <bzg@gnu.org> wrote:

> Hi all,
>
> thanks to joint efforts by Christian Köstlin and Ihor, we are running
> the Org test suite from the Org main branch against Emacs 26, 27, 28.
>

It's great to hear this. Thank you, Christian and Ihor!

[-- Attachment #2: Type: text/html, Size: 589 bytes --]