From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id wBe+BZ1D62SuIgAAauVa8A:P1 (envelope-from ) for ; Sun, 27 Aug 2023 14:37:49 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id wBe+BZ1D62SuIgAAauVa8A (envelope-from ) for ; Sun, 27 Aug 2023 14:37:49 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [IPv6:2a01:4f9:c011:7a79::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 409435F283 for ; Sun, 27 Aug 2023 14:37:48 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=boula.in header.s=key1 header.b=NeaQYWaE; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org; dmarc=fail reason="SPF not aligned (relaxed)" header.from=boula.in (policy=quarantine) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1693139869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=3hBxUB69ljq/MRnxVot4PAY9x20/hrsihja+G7Dba3Q=; b=pUrbPlyMKLkupgxN7W/J/hbEE9OXlYl1Ohy/DH/ZG2AZwTmFSnrAcqSXq8hsLrx4KrFgkd MjZNO65Knc1O8lbokXYpJKGK8hzKKUQQFp5vP/GrObkihTWW6p9HW4YNtSgP1P66HGty1j QWtdcFZ3utnAEsErKZ8+tunfwIon/AO8rf0n3pHDe9FEflprXbK5inrOenNjFlJJAI0XyP 3gPse9xbGe9+Vg9rQ09Jsy2aTuARWaQBxYJ/TliSUeUUbUV412vaxtvXH3BqxjejC8a5VO fSVfBp7SmyyJ7mi9aXY2pnQWPZYwiYZJN4/qryKUrwymaoOOHfBkyFf5g/u/vw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1693139869; a=rsa-sha256; cv=none; b=bK+EYwGL+1smddpb51zAXnsRf7SUvTQ/k7+AeTDiIsMSXbwA5K1FnoO0BvAdn7F+2zYW6W bE1LeBImkUc059EIUNlrrm8TuYaymUJOD925QbvOEy75KH0/p0fSXwBXkkKxUY7X3Lgc/Y NGdnhIQXEQ7mqBOb6I9hLFLq5V6tbJkfmgzJx9ZwseOGx6Dl2grzQMJAiVs3ZNGW9G0MjT R44LsKQ3ALTWMYnx06nWm/L0VEmVlYSyAgKooIaFhzCVDSwar3B7oJTvDurlm19V9cOD9K B6qp7/90HJrX8bWUOECT8ruZt49Vt5q6hQpSGs3hSXMA+CLzYBBSbQfQ8ZWthA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=boula.in header.s=key1 header.b=NeaQYWaE; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org; dmarc=fail reason="SPF not aligned (relaxed)" header.from=boula.in (policy=quarantine) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id B9BB85F362; Sun, 27 Aug 2023 12:37:33 +0000 (UTC) Received: from out-243.mta1.migadu.com (out-243.mta1.migadu.com [IPv6:2001:41d0:203:375::f3]) by mail.notmuchmail.org (Postfix) with ESMTPS id 63B2F5E00E for ; Sun, 27 Aug 2023 12:37:30 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=boula.in; s=key1; t=1693139492; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=spcOZTE/6HSv2LUtXzjyT4WN1xZMTIcW1WVISi67MMk=; b=NeaQYWaE85Q73QZmW1B3K1HL4Ue3774YRwbzE9hML7jacGj/99ZvlhaLeKDRb63Dlq0qCi 6lqTHn0eplgOk9Vg+jQrag3xzSphiRA9m7YeK1YYo5sCVdmhJIyh0in+2yYkB51n/QMNLQ WQ5zd5vjUXNBtdRKd0VQZdpdLD7kJD4EXFJVwWIxMqhtDUdZbkgNjjwPh0wK0f2iGFiG/6 3Gxik5k0UQPKLUnNNvwxrRibVvx5RupsVCaUVyqzckpl9gn2WM5MIaLm2v2++p374WmBGs UKPRsLwaR4gVLog6Vugkr4KxoUMSWn+ff0UdDojjeXf+QNDszjPvSw6UTvZ+rw== From: Kevin Boulain To: notmuch@notmuchmail.org Subject: [PATCH v2 1/2] test: showcase thread-unsafe s-expression query parser Date: Sun, 27 Aug 2023 14:31:02 +0200 Message-ID: <20230827123103.22744-1-kevin@boula.in> In-Reply-To: <87fs5fg40h.fsf@tethera.net> References: <87fs5fg40h.fsf@tethera.net> MIME-Version: 1.0 Message-ID-Hash: WJG6UNFONKKCUORLBZL3F7MCWGGZBTHG X-Message-ID-Hash: WJG6UNFONKKCUORLBZL3F7MCWGGZBTHG X-MailFrom: kevin@boula.in X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0 CC: Kevin Boulain X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-Country: DE X-Spam: Yes X-Migadu-Spam-Score: 6.00 X-Spam-Score: 6.00 X-Migadu-Queue-Id: 409435F283 X-Migadu-Spam: Yes X-Migadu-Scanner: mx1.migadu.com X-TUID: OpEgGVFxvFXF The test fails reliably when Notmuch is compiled as such: ./configure make CFLAGS=-fsanitize=thread LDFLAGS=-fsanitize=thread It's still unclear how the test suite could be run with the correct compilation flags. Output: T810-tsan: Testing run code with TSan enabled against the library PASS create PASS query FAIL sexp query --- T810-tsan.3.EXPECTED 2023-04-03 19:53:04.400771102 +0000 +++ T810-tsan.3.OUTPUT 2023-04-03 19:53:04.402771109 +0000 @@ -1,2 +1,44 @@ == stdout == == stderr == +================== +WARNING: ThreadSanitizer: data race (pid=21372) + Read of size 4 at 0x7b1000000188 by thread T2: + #0 Xapian::Internal::intrusive_ptr::intrusive_ptr(Xapian::Internal::intrusive_ptr const&) .../xapian/intrusive_ptr.h:107 (libnotmuch.so.5+0x3017b) + #1 Xapian::Query::Query(Xapian::Query const&) .../xapian/query.h:328 (libnotmuch.so.5+0x2fcbe) + #2 _sexp_to_xapian_query lib/parse-sexp.cc:707 (libnotmuch.so.5+0x43f1f) + #3 _notmuch_sexp_string_to_xapian_query(_notmuch_database*, char const*, Xapian::Query&) lib/parse-sexp.cc:729 (libnotmuch.so.5+0x44207) + #4 _notmuch_query_ensure_parsed_sexpr lib/query.cc:240 (libnotmuch.so.5+0x2c59e) + #5 _notmuch_query_ensure_parsed lib/query.cc:258 (libnotmuch.so.5+0x2c646) + #6 _notmuch_query_search_documents lib/query.cc:362 (libnotmuch.so.5+0x2cc0e) + #7 notmuch_query_search_messages lib/query.cc:350 (libnotmuch.so.5+0x2cb76) + #8 thread CWD/test2.c:17 (test2+0x4012f4) + + Previous write of size 4 at 0x7b1000000188 by thread T1: + #0 Xapian::Internal::intrusive_ptr::intrusive_ptr(Xapian::Internal::intrusive_ptr const&) .../xapian/intrusive_ptr.h:107 (libnotmuch.so.5+0x3018e) + #1 Xapian::Query::Query(Xapian::Query const&) .../xapian/query.h:328 (libnotmuch.so.5+0x2fcbe) + #2 _sexp_to_xapian_query lib/parse-sexp.cc:707 (libnotmuch.so.5+0x43f1f) + #3 _notmuch_sexp_string_to_xapian_query(_notmuch_database*, char const*, Xapian::Query&) lib/parse-sexp.cc:729 (libnotmuch.so.5+0x44207) + #4 _notmuch_query_ensure_parsed_sexpr lib/query.cc:240 (libnotmuch.so.5+0x2c59e) + #5 _notmuch_query_ensure_parsed lib/query.cc:258 (libnotmuch.so.5+0x2c646) + #6 _notmuch_query_search_documents lib/query.cc:362 (libnotmuch.so.5+0x2cc0e) + #7 notmuch_query_search_messages lib/query.cc:350 (libnotmuch.so.5+0x2cb76) + #8 thread CWD/test2.c:17 (test2+0x4012f4) + + Location is heap block of size 56 at 0x7b1000000180 allocated by main thread: + #0 operator new(unsigned long) (libtsan.so.2+0x8ba83) + #1 Xapian::Query::Query(std::__cxx11::basic_string, std::allocator > const&, unsigned int, unsigned int) (libxapian.so.30+0x9f200) + #2 __static_initialization_and_destruction_0(int, int) (libxapian.so.30+0xa27ac) + #3 _GLOBAL__sub_I_query.cc (libxapian.so.30+0xa286d) + #4 call_init (ld-linux-x86-64.so.2+0x5e1d) + + Thread T2 (tid=21375, running) created by main thread at: + #0 pthread_create (libtsan.so.2+0x62de6) + #1 main CWD/test2.c:24 (test2+0x4013ba) + + Thread T1 (tid=21374, running) created by main thread at: + #0 pthread_create (libtsan.so.2+0x62de6) + #1 main CWD/test2.c:23 (test2+0x401380) + +SUMMARY: ThreadSanitizer: data race .../xapian/intrusive_ptr.h:107 in Xapian::Internal::intrusive_ptr::intrusive_ptr(Xapian::Internal::intrusive_ptr const&) +================== +ThreadSanitizer: reported 1 warnings --- test/T810-tsan.sh | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/test/T810-tsan.sh b/test/T810-tsan.sh index 7e877b27..de98c0a9 100755 --- a/test/T810-tsan.sh +++ b/test/T810-tsan.sh @@ -4,7 +4,8 @@ test_directory=$(cd "$(dirname "${BASH_SOURCE[0]}")" > /dev/null && pwd) test_description='run code with TSan enabled against the library' # Note it is hard to ensure race conditions are deterministic so this -# only provides best effort detection. +# only provides best effort detection. Compile Notmuch with +# make CFLAGS=-fsanitize=thread LDFLAGS=-fsanitize=thread . "$test_directory"/test-lib.sh || exit 1 @@ -89,4 +90,44 @@ cat < EXPECTED EOF test_expect_equal_file EXPECTED OUTPUT +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + test_begin_subtest "sexp query" + test_subtest_known_broken + test_C ${MAIL_DIR} ${MAIL_DIR}-2 < +#include + +void *thread (void *arg) { + char *mail_dir = arg; + notmuch_database_t *db; + /* + * Query generation from s-expression used the tread-unsafe + * Xapian::Query::MatchAll. + */ + EXPECT0(notmuch_database_open_with_config (mail_dir, + NOTMUCH_DATABASE_MODE_READ_ONLY, + NULL, NULL, &db, NULL)); + notmuch_query_t *query; + EXPECT0(notmuch_query_create_with_syntax (db, "(from *)", NOTMUCH_QUERY_SYNTAX_SEXP, &query)); + notmuch_messages_t *messages; + EXPECT0(notmuch_query_search_messages (query, &messages)); + return NULL; +} + +int main (int argc, char **argv) { + pthread_t t1, t2; + EXPECT0(pthread_create (&t1, NULL, thread, argv[1])); + EXPECT0(pthread_create (&t2, NULL, thread, argv[2])); + EXPECT0(pthread_join (t1, NULL)); + EXPECT0(pthread_join (t2, NULL)); + return 0; +} +EOF + cat < EXPECTED +== stdout == +== stderr == +EOF + test_expect_equal_file EXPECTED OUTPUT +fi + test_done