From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id gNWbHc43K2TZWAAASxT56A (envelope-from ) for ; Mon, 03 Apr 2023 22:32:14 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id kN6gHc43K2RaOgEA9RJhRA (envelope-from ) for ; Mon, 03 Apr 2023 22:32:14 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1E5F83556C for ; Mon, 3 Apr 2023 22:32:14 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id DEC185F7E9; Mon, 3 Apr 2023 20:32:11 +0000 (UTC) Received: from out-58.mta1.migadu.com (out-58.mta1.migadu.com [IPv6:2001:41d0:203:375::3a]) by mail.notmuchmail.org (Postfix) with ESMTPS id B565E5E53D for ; Mon, 3 Apr 2023 20:32:08 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=boula.in; s=key1; t=1680553927; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=791ow75CZZDhAI38iDRAKOKFdcQXMjqeW0FlDuYNwcg=; b=VEOpg6rsMPsolKAHRcMvHIhUQB54OeHCsb613g5UlEWGLxy/nA93VA3sDtvUoK42JE3w29 j7qKn30Hm7+IfmAiUdpW/VPbrqnkSb3PbOAM7tHBSE3rSAPtUhzzRPIovosegwuJn6CyPs 3acwH1rwnu4Hkb98dwQqAaDIkXhbgz2r1j73mfRamKSJoCJYlienoU+LBJaHmFiCNcQJYf 0Pk3n7nXGKBcgIX4cbnsBiVIeaFoLJn/GPDcHe9K8vfjI34Qk9hysp/lH2T7gFmk+ebBGs ZvtC85unbR1F9SPXwpXYT1+9gFJuP8W+VxpqaozAvld4F1KRvu4psKmlaq99hw== From: Kevin Boulain To: notmuch@notmuchmail.org Subject: [PATCH 1/2] test: showcase thread-unsafe s-expression query parser Date: Mon, 3 Apr 2023 22:31:45 +0200 Message-Id: <20230403203146.39749-1-kevin@boula.in> MIME-Version: 1.0 Message-ID-Hash: 4KAGXUHY6MFVAT3S6HPWOVPNB4CKRCBY X-Message-ID-Hash: 4KAGXUHY6MFVAT3S6HPWOVPNB4CKRCBY X-MailFrom: kevin@boula.in X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0 CC: Kevin Boulain X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Country: DE X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1680553934; a=rsa-sha256; cv=none; b=lYeSE761MKjUshyI2xEIdGHBFkjQR5VhMy7/faMhIog5TpYHFYrMTGojTQBCgad/KfwVy9 o0MVIe62k3dxI5Aka+aCmYqap8XEwTwKAqKmjxsPiLEDy3IMZDtZ+MiiLepnrvKt5FkaDO SwXCJi6v+Se7f3pJBYhyUBq9qLVdnMmEkvgfG5JAvPs3UI4BlBTSfD7fyyGjqzKcF3Ktnp 65j40QZ7DJFPfgAJC2AIzSiizegRUWHO815mVBrEn9p5lBS+xkxtRmsOWMgEY8AHjvQTlQ TRIqpA4p2bJeSza6BKzSpwuRch+ar+24WANxs3KPCTwxVrGQ8DNdG+hiiDKdoA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=boula.in header.s=key1 header.b=VEOpg6rs; dmarc=fail reason="SPF not aligned (relaxed)" header.from=boula.in (policy=quarantine); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1680553934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=aKAElI3eW/B4eD9fNvk+Yl9/KiSABavQKhdLAmZny+Y=; b=X4PxTikFxGHCasD0PJrWtPx2Gt11KBjvQaNY3EJchIYz7xMWpXDM9zbvVJlYTUBQ/miZUO S0MEMVhEri9Acmkkl6Zwf8ohG5Y6p47qHzT/juQocdy4vgi/oe2ZkfsjOAFI7WLB7oW4Ri Kv8tgmRFn2DEtWnj4iZpZSvXHYp0ETXs8x+51fX+FtPZrZKBAauHc5ZzTQWr0LHhzHARKz QEbcDPegfnXr/y/cN74Rvn/JvsH2qLNmL08o/FbIXDlzeoYPrW1BHeyjCkQrtDNHiNGcHX +8g/X8a8sc0M0JPQaKx3Jf5wMvE0N5La2dod/1Cz0OsTLfGlC0ct3oVXlngdvQ== X-Migadu-Spam-Score: 6.00 X-Spam-Score: 6.00 X-Migadu-Queue-Id: 1E5F83556C X-Migadu-Spam: Yes Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=boula.in header.s=key1 header.b=VEOpg6rs; dmarc=fail reason="SPF not aligned (relaxed)" header.from=boula.in (policy=quarantine); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Scanner: scn1.migadu.com X-Spam: Yes X-TUID: byPniBwYfktT The test fails quite reliably for me: T810-tsan: Testing run code with TSan enabled against the library PASS create PASS query FAIL sexp query --- T810-tsan.3.EXPECTED 2023-04-03 19:53:04.400771102 +0000 +++ T810-tsan.3.OUTPUT 2023-04-03 19:53:04.402771109 +0000 @@ -1,2 +1,44 @@ == stdout == == stderr == +================== +WARNING: ThreadSanitizer: data race (pid=21372) + Read of size 4 at 0x7b1000000188 by thread T2: + #0 Xapian::Internal::intrusive_ptr::intrusive_ptr(Xapian::Internal::intrusive_ptr const&) .../xapian/intrusive_ptr.h:107 (libnotmuch.so.5+0x3017b) + #1 Xapian::Query::Query(Xapian::Query const&) .../xapian/query.h:328 (libnotmuch.so.5+0x2fcbe) + #2 _sexp_to_xapian_query lib/parse-sexp.cc:707 (libnotmuch.so.5+0x43f1f) + #3 _notmuch_sexp_string_to_xapian_query(_notmuch_database*, char const*, Xapian::Query&) lib/parse-sexp.cc:729 (libnotmuch.so.5+0x44207) + #4 _notmuch_query_ensure_parsed_sexpr lib/query.cc:240 (libnotmuch.so.5+0x2c59e) + #5 _notmuch_query_ensure_parsed lib/query.cc:258 (libnotmuch.so.5+0x2c646) + #6 _notmuch_query_search_documents lib/query.cc:362 (libnotmuch.so.5+0x2cc0e) + #7 notmuch_query_search_messages lib/query.cc:350 (libnotmuch.so.5+0x2cb76) + #8 thread CWD/test2.c:17 (test2+0x4012f4) + + Previous write of size 4 at 0x7b1000000188 by thread T1: + #0 Xapian::Internal::intrusive_ptr::intrusive_ptr(Xapian::Internal::intrusive_ptr const&) .../xapian/intrusive_ptr.h:107 (libnotmuch.so.5+0x3018e) + #1 Xapian::Query::Query(Xapian::Query const&) .../xapian/query.h:328 (libnotmuch.so.5+0x2fcbe) + #2 _sexp_to_xapian_query lib/parse-sexp.cc:707 (libnotmuch.so.5+0x43f1f) + #3 _notmuch_sexp_string_to_xapian_query(_notmuch_database*, char const*, Xapian::Query&) lib/parse-sexp.cc:729 (libnotmuch.so.5+0x44207) + #4 _notmuch_query_ensure_parsed_sexpr lib/query.cc:240 (libnotmuch.so.5+0x2c59e) + #5 _notmuch_query_ensure_parsed lib/query.cc:258 (libnotmuch.so.5+0x2c646) + #6 _notmuch_query_search_documents lib/query.cc:362 (libnotmuch.so.5+0x2cc0e) + #7 notmuch_query_search_messages lib/query.cc:350 (libnotmuch.so.5+0x2cb76) + #8 thread CWD/test2.c:17 (test2+0x4012f4) + + Location is heap block of size 56 at 0x7b1000000180 allocated by main thread: + #0 operator new(unsigned long) (libtsan.so.2+0x8ba83) + #1 Xapian::Query::Query(std::__cxx11::basic_string, std::allocator > const&, unsigned int, unsigned int) (libxapian.so.30+0x9f200) + #2 __static_initialization_and_destruction_0(int, int) (libxapian.so.30+0xa27ac) + #3 _GLOBAL__sub_I_query.cc (libxapian.so.30+0xa286d) + #4 call_init (ld-linux-x86-64.so.2+0x5e1d) + + Thread T2 (tid=21375, running) created by main thread at: + #0 pthread_create (libtsan.so.2+0x62de6) + #1 main CWD/test2.c:24 (test2+0x4013ba) + + Thread T1 (tid=21374, running) created by main thread at: + #0 pthread_create (libtsan.so.2+0x62de6) + #1 main CWD/test2.c:23 (test2+0x401380) + +SUMMARY: ThreadSanitizer: data race .../xapian/intrusive_ptr.h:107 in Xapian::Internal::intrusive_ptr::intrusive_ptr(Xapian::Internal::intrusive_ptr const&) +================== +ThreadSanitizer: reported 1 warnings --- test/T810-tsan.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/test/T810-tsan.sh b/test/T810-tsan.sh index 7e877b27..c9008c6b 100755 --- a/test/T810-tsan.sh +++ b/test/T810-tsan.sh @@ -89,4 +89,44 @@ cat < EXPECTED EOF test_expect_equal_file EXPECTED OUTPUT +if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then + test_begin_subtest "sexp query" + test_subtest_known_broken + test_C ${MAIL_DIR} ${MAIL_DIR}-2 < +#include + +void *thread (void *arg) { + char *mail_dir = arg; + notmuch_database_t *db; + /* + * Query generation from s-expression used the tread-unsafe + * Xapian::Query::MatchAll. + */ + EXPECT0(notmuch_database_open_with_config (mail_dir, + NOTMUCH_DATABASE_MODE_READ_ONLY, + NULL, NULL, &db, NULL)); + notmuch_query_t *query; + EXPECT0(notmuch_query_create_with_syntax (db, "(from *)", NOTMUCH_QUERY_SYNTAX_SEXP, &query)); + notmuch_messages_t *messages; + EXPECT0(notmuch_query_search_messages (query, &messages)); + return NULL; +} + +int main (int argc, char **argv) { + pthread_t t1, t2; + EXPECT0(pthread_create (&t1, NULL, thread, argv[1])); + EXPECT0(pthread_create (&t2, NULL, thread, argv[2])); + EXPECT0(pthread_join (t1, NULL)); + EXPECT0(pthread_join (t2, NULL)); + return 0; +} +EOF + cat < EXPECTED +== stdout == +== stderr == +EOF + test_expect_equal_file EXPECTED OUTPUT +fi + test_done