Re: [PATCH 3/3] test: ruby: simplify basic tests

[Felipe Contreras <felipe.contreras@gmail.com>]

On Mon, Jun 7, 2021 at 6:53 PM David Bremner <david@tethera.net> wrote:
>
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
> >>
> >> Is this assuming that the sort order in the CLI is the same as in the
> >> library / bindings? that seems a bit fragile if so.
> >
> > Both the CLI and the bindings are using the same libnotmuch library.
> > If neither of them specify a sort order, the default sort order of
> > libnotmuch would be used (I presume). Exactly the same order I would
> > get if I write a C program that uses libnotmuch and doesn't specify
> > any order.
> >
> > Why would the CLI specify an order the user didn't specify to libnotmuch?
>
> I guess the point is that the CLI is not required to track the library
> precisely, so even if it is a bit theoretical, this change does
> introduce some fragility / technical debt into the test suite.
>
> For better or for worse, the API does not document a default sort order,
> so assuming any particular sort order is probably a mistake. I can
> imagine a future database backend where the "UNSORTED" order is actually
> non-deterministic. If we were to document a default sort order, UNSORTED
> might make the most sense as a default, as it's the highest performance
> (more or less by definition).

We could have test_ruby_unordered and sort both the expected and
actual files to ensure the tests don't break if in the future the
default order changes. But personally I don't see much point in
complicating the tests preemptively. I would rather worry when the
time comes, which might be never.

-- 
Felipe Contreras