From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2CBkJbE3vmAybwEAgWs5BA (envelope-from ) for ; Mon, 07 Jun 2021 17:13:53 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id uBG2ILE3vmAIaAAA1q6Kng (envelope-from ) for ; Mon, 07 Jun 2021 15:13:53 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0376F24DCA for ; Mon, 7 Jun 2021 17:13:53 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id B5BD729C43; Mon, 7 Jun 2021 11:13:44 -0400 (EDT) Received: from lahtoruutu.iki.fi (lahtoruutu.iki.fi [185.185.170.37]) by mail.notmuchmail.org (Postfix) with ESMTPS id BBFE61FFD8 for ; Mon, 7 Jun 2021 11:13:41 -0400 (EDT) Received: from guru.guru-group.fi (unknown [IPv6:2a02:2380:1:9:5054:ff:feb7:a4bc]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: too) by lahtoruutu.iki.fi (Postfix) with ESMTPSA id 386441B00334; Mon, 7 Jun 2021 18:13:38 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1623078818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8uuopipMiFIC5zZMLpWxrSRCqOrEjLIaucEHQuWR6I8=; b=IAOfNtHa3Kim+s7mnXBP5cg00O5GTK1+7roHKHH1AKJjIooyaPt8xwG9KAjzWggOHpnfi6 qbaaA9fwa8SscmXLFXj0PzjgiqXwDP4P/opqdXWGuEZACvp3dEOlQTOkdfcLCsd7YkUrjw HSdr8w5UfQGRiRXnu5SgUbgPFaRLrXl2gCcI/UhNpF+gXijvF8zr8Jm5xGPJO0cL1UxJ4j oh/oLEeqbV1O6Rib8JVGRzvhnV9ylQ5NG+kfdgUXijiZrhYTxJ+cIlYzXLS+YlAY3PX6lZ W32vskPi/qftofa6DgfPVatuzPPsld/US2wT68+HTeHRz7e0CwWJiniJJp3dLw== From: Tomi Ollila To: David Bremner , notmuch@notmuchmail.org Subject: Re: [PATCH] test: aggregate-results updates In-Reply-To: <87y2boimzg.fsf@tethera.net> References: <20210517081109.31897-1-tomi.ollila@iki.fi> <87y2boimzg.fsf@tethera.net> User-Agent: Notmuch/0.32+21~gfdddd4c (https://notmuchmail.org) Emacs/27.1 X-Face: HhBM'cA~ MIME-Version: 1.0 ARC-Authentication-Results: i=1; ORIGINATING; auth=pass smtp.auth=too smtp.mailfrom=tomi.ollila@iki.fi ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1623078818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8uuopipMiFIC5zZMLpWxrSRCqOrEjLIaucEHQuWR6I8=; b=n+32+LKvjNS+/3ZZDvHGfvscX4ED343X8TR6cz/drIiMudNEeKTrMw3mmQzjhZUQvH+sPS GvoBkliZ1YcBQvSzaPurgs+PW/GaYNkfXMiqiyhXJMUsPsS0kdd71KbcchQeSyYhaKeABG CM/wPxUmSjFnJ8AFvD83bfATX2unZl8JqTqP9klsHgzwXRrvYNaT+LICdDYVVSdl6zrGL2 b94RsLcD9h9FyjN0Kg9uA33gkw80UKzhWLL7IV/32OIjAGOzHjkYhjo/o1JUQ1lY5OAJnZ yQ3LHfX4qQx+K3b7RlqwBVEpWvnFadXlA+gTYVCZGSYZ6EFCDBrOWuw7C/p/gw== ARC-Seal: i=1; s=lahtoruutu; d=iki.fi; t=1623078818; a=rsa-sha256; cv=none; b=lo1Fr1GC0Gq9kN0iNiw5lk9FUVyFHpGg6JtNj2r0jIXqtWhBXE3PLvIZsemG6y2F0ARguY ytWP/Uj8LSHJ+c6fvRi4y9TtTPS3Bh4yxJyXKkEh3MFpYI+G9nAO6F+r5+80e7oD4kjSgR Deqv/kG9ls2DYWi7YIjsypXqtSlUlDTOsWtH7NzgOdwhJpnLUeI5YXtOWWh08DpxkaANq7 ufIQvf0/A+azt6KBwI5UTuNkOxQGbMoiaKhLNw4fbc9ydFdas4XDDk+DW0PG9EYifoF8CI q+oqdtU5BULK6RGo5eJuBD7iiut11tVvahIDvxVndshBRSth0QJmclCVDWrbRg== Message-ID-Hash: C3NICCEV45DYHO6VSUNGDBCMDWK4CFPA X-Message-ID-Hash: C3NICCEV45DYHO6VSUNGDBCMDWK4CFPA X-MailFrom: tomi.ollila@iki.fi X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1623078833; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=LVoKbeM2vDISIcI7ecTy8Me8dzY3YeY2Xh2nKqh5Tu4=; b=VuBxbyoJau8Tu9df8lJ37smgzXdoK09znuUnzweVJzOISIfCN1NgHUDrfK2wJlFbjl8qKD z/5qfzZpBDrlirCLWh4buI1AHGeW5nGrUmj6Rf6l6fCbs8BtChj/PxM6AH6PWOk7H0H7lF ymKYLZVDj8PQ98wHFXyzxSIttiEPDkVNvhbgPMbrCUiJvjgsto5DMPZcemfTbZvVsn5Uyb WY6gokBjD2nbfosM0bsYU5mMA2+UAxtKQcz862MEya6tbWNYxxgp4aCpkpNwiFGzpMB6Bo hucvl4gbeFTrznf978vFzrE/fsTWVJRQOhf9ooZbKbSBSHYzVojI0EGNCplzBw== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1623078833; a=rsa-sha256; cv=fail; b=d2VBVh1VCHmz86VnPyeIRGqJhWY8ACHZPvIv7F4k6cqd6Pb8PQOXFHKKRRtTE/mt7N2ja8 pf6XgbwA6HOZm6j/5l+6OTslO49SyKgJBnwKRBPUqkJvHB854DNqQtlscoZ7gZE7JQ/Nuk 2KccOjhyRUOGeJBqUpATRDPn55J07jGNnbUx6eGQB7G9sfxX8avQRRJDLQWqka7geE7kwF ATPsffqXcHwiO87UCPPUtCgnmICbdMAxq3TRUQ8e7ItjWrV9NATF8S+P4tHk9r8L/HjPkQ 1vIu+PWMHOEUdWAhchSN6PzrxdMOtwow1iHn3uEaMKxoqKC1FQDQZ6gRLfD5Ig== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=lahtoruutu header.b=IAOfNtHa; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 0.97 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=lahtoruutu header.b=IAOfNtHa; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 0376F24DCA X-Spam-Score: 0.97 X-Migadu-Scanner: scn0.migadu.com X-TUID: oBsD4F8iez3x On Sat, Jun 05 2021, David Bremner wrote: > Tomi Ollila writes: >> >> testing all changed branches is somewhat hard; I did by adding >> sleep 20 and file what effective removed created results files; >> another change was to change 'total' to 'toxtal' in test_done, >> then I run (at least) the following command lines: > > changing test_done results in pretty nonsensical output. I guess that's > OK? Yes. It is very unlikely that if something is written to file not all data were written (as the amount of data to be written is low). A bit less unlikely is the case that output file is created but no data written. Should such an extremely rare cases happen it should be enough just to report it so it get noticed (and not add any extra code to handle such cases any further). One checking the results would probably guess that if output is as nonsensical as it is something went badly wrong. Tomi > > $ NOTMUCH_TESTS=T310-emacs.sh ./notmuch-test > > ... > > '/home/bremner/software/upstream/notmuch/test/test-results/T310-emacs' lacks 'total ...'; results may be inconsistent. > Notmuch test suite complete. > 72/0 tests passed. > 1 test failed. > -73 tests skipped. > _______________________________________________ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-leave@notmuchmail.org