T568-lib-thread is non-deterministic

T568-lib-thread is non-deterministic

[Alyssa Ross]

T568-lib-thread assums that there will always be a thread with ID
thread:0000000000000009, but this turns out not to be the case.

This was reported to us in Nixpkgs[1], where it caused builds of our
notmuch package to intermittently fail, and I was able to
(intermittently) reproduce.  We are going to work around this problem
for now by disabling T568.  I can provide a Xapian DB generated by this
test with no thread with that ID if that's helpful.  I believe this also
explains this report previously sent to the Notmuch mailing list[2].

Note on reproducing: I found that it often would succeed repeatedly
many, many times before failing, and then failing consistently for a
while.  My co-conspirator Puck (CCed) thinks this might be due to how
directory order works in ZFS (which just happens to be the filesystem we
are using).

[1]: https://github.com/NixOS/nixpkgs/issues/109092
[2]: https://nmbug.notmuchmail.org/nmweb/show/874ko8trhk.fsf%40tethera.net

Re: T568-lib-thread is non-deterministic

[Alyssa Ross]

[-- Attachment #1.1: Type: text/plain, Size: 1114 bytes --]

On Thu, Jan 14, 2021 at 03:59:20PM +0000, Alyssa Ross wrote:
> T568-lib-thread assums that there will always be a thread with ID
> thread:0000000000000009, but this turns out not to be the case.
>
> This was reported to us in Nixpkgs[1], where it caused builds of our
> notmuch package to intermittently fail, and I was able to
> (intermittently) reproduce.  We are going to work around this problem
> for now by disabling T568.  I can provide a Xapian DB generated by this
> test with no thread with that ID if that's helpful.  I believe this also
> explains this report previously sent to the Notmuch mailing list[2].
>
> Note on reproducing: I found that it often would succeed repeatedly
> many, many times before failing, and then failing consistently for a
> while.  My co-conspirator Puck (CCed) thinks this might be due to how
> directory order works in ZFS (which just happens to be the filesystem we
> are using).
>
> [1]: https://github.com/NixOS/nixpkgs/issues/109092
> [2]: https://nmbug.notmuchmail.org/nmweb/show/874ko8trhk.fsf%40tethera.net

CCing Puck with the correct email address now.  Whoops.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: T568-lib-thread is non-deterministic

[David Bremner]

Alyssa Ross <hi@alyssa.is> writes:

> T568-lib-thread assums that there will always be a thread with ID
> thread:0000000000000009, but this turns out not to be the case.
>
> This was reported to us in Nixpkgs[1], where it caused builds of our
> notmuch package to intermittently fail, and I was able to
> (intermittently) reproduce.  We are going to work around this problem
> for now by disabling T568.  I can provide a Xapian DB generated by this
> test with no thread with that ID if that's helpful.  I believe this also
> explains this report previously sent to the Notmuch mailing list[2].
>
> Note on reproducing: I found that it often would succeed repeatedly
> many, many times before failing, and then failing consistently for a
> while.  My co-conspirator Puck (CCed) thinks this might be due to how
> directory order works in ZFS (which just happens to be the filesystem we
> are using).

Sorry for the delay in replying. This should be fixed in commit a37d5f5,
which was part of Notmuch 0.31.4.

d