From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 4AQRKLqkMmA+MwAA0tVLHw (envelope-from ) for ; Sun, 21 Feb 2021 18:21:46 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id gED4I7qkMmDKCAAA1q6Kng (envelope-from ) for ; Sun, 21 Feb 2021 18:21:46 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 07A721AE5C for ; Sun, 21 Feb 2021 19:21:43 +0100 (CET) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 957FF26BBE; Sun, 21 Feb 2021 13:21:34 -0500 (EST) Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by mail.notmuchmail.org (Postfix) with ESMTPS id 26AFE26ACB for ; Sun, 21 Feb 2021 13:21:32 -0500 (EST) Received: by mail-pj1-x1033.google.com with SMTP id gm18so1915632pjb.1 for ; Sun, 21 Feb 2021 10:21:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ljsNyXLe3Ka1olhSvCZSg3G901itjf8acVaXifU2Si4=; b=jNf3frnPnQ3tDPK21y1Ew/ItYDQZDcvQ/D100Tfrwp4lH1Gb//5SIiM/ugrE6XtEvd uNpiC/YK36Ckli7Q969gVZ+Y8/KjYIb9Un3EJZ6E+TurKZVE54DGqA97GPNzQiTEU3hd I1OEJxK4o8Rf9WOt6HU3W0rp6lWaUof9S5Kxj9RxhDJt+9AUTCbyo9pWXP5WeS6Xs3gO IxlfmdblJZN9PClxiB3TVEeqCiQ4nA+oZDIrmX6vXAx9QFMVxMY4VLpJAqWWMxOCoxtU PSjHTl6LaV+PCFCi6k9ypdF+Wo1Fotc3fc0wz2RefLdoYuVLtwWVa0mcRVFQ6FRoAoZV yugg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ljsNyXLe3Ka1olhSvCZSg3G901itjf8acVaXifU2Si4=; b=EM4fFbj462Sfc1Fs0xy+O9m1Bu87CAIN+dLb/B5FcBb6gVVW2/YqtZgOC84nsW9HRN uVese4guX2jpgWAMJKx64GpxzPccCs/IGK496T+2CRTvfdW50hq5L+KXL5oPu2wlMohj WAQINtiIJW1svZgh9N1VWxruZplBPho5XrMB6XLSAoxc7I9VjZgWW1L9Ywtixep7ZxBt SkgUAaZ/UK979XLbgWyqQK8HVcfL9N0DrVgNN4fAjTbuDSz3PbBGBq5KZWtjvVQw6EdS 4ItwbgCgjRnFqGZoeNgSmJprZecwKa7aR0mBfDJwWPnbrNEUDSh8FOIz785Xp2Nawn40 kX+A== X-Gm-Message-State: AOAM531KdtjjKsZ0JmuQOjKVhdPEMIYrLNuCmGVIw/Ls49Fp4V9Dix4p 5KfqdL1sBba+mHec6qPVMp6ZTC3ishAmFXWwa0b57aNg X-Google-Smtp-Source: ABdhPJzzn2cdKNSRS4CcElaRvx8X+tsdcTvta6HO9NbSq73J1LQdC4xX6aNiEQZHghoWJfKM4TD1K1BpxCa+nqhj8iA= X-Received: by 2002:a17:90a:609:: with SMTP id j9mr19742422pjj.198.1613931687605; Sun, 21 Feb 2021 10:21:27 -0800 (PST) MIME-Version: 1.0 References: <87wnv4qm7s.fsf@tethera.net> In-Reply-To: <87wnv4qm7s.fsf@tethera.net> From: Xu Wang Date: Sun, 21 Feb 2021 13:21:16 -0500 Message-ID: Subject: Re: parallel test failures To: David Bremner Message-ID-Hash: SK6XFMIMJDURT3IKB5IINUNIMYYQAMCB X-Message-ID-Hash: SK6XFMIMJDURT3IKB5IINUNIMYYQAMCB X-MailFrom: xuwang762@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: notmuch@notmuchmail.org X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: 0.71 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20161025 header.b=jNf3frnP; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 07A721AE5C X-Spam-Score: 0.71 X-Migadu-Scanner: scn1.migadu.com X-TUID: 1IEXbuXDHJwy I did not look at logs, but I have had problem in other scenarios. The way I debugged was to use strace to get a list of all files the tests accessed. From that list I could recognize that some files that should have been in separate temp directories were not thread-specific and solution was to put the temp files in separate dir for each test. Not sure if this is helpful, but wanted to share. Kind regards and best of luck, Xu On Fri, Feb 19, 2021 at 7:24 AM David Bremner wrote: > > > I have intermittent failures when running the test suite on sufficiently > parallel machines. I have attached a log of such a failing build, > although it does not seem especially illuminating. > > It takes anywhere from 5 to 300 runs to get a failure for me running on > 60 hardware threads (30 cores). At least on this machine the number of > tests that pass seems consistent at 1205 > > _______________________________________________ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-leave@notmuchmail.org