From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id A0A4A6DE102D for ; Sun, 5 May 2019 08:26:20 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.156 X-Spam-Level: X-Spam-Status: No, score=-0.156 tagged_above=-999 required=5 tests=[AWL=0.045, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KS-x8ZyLCOv5 for ; Sun, 5 May 2019 08:26:20 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTPS id CCF3C6DE0F9A for ; Sun, 5 May 2019 08:26:19 -0700 (PDT) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1557069978; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=PNgmQPvZ0m9b5E6+AqDfsuycGtwDzEFLFx5FPW6Pg5A=; b=7PMwvshocSOfKXaiAy2m0jfBwWPAenum5nyo6WUbxU1QiRx6pyS3KG6Z Sv6T9Z+fsQN1AOli0v7MLzXXK5YaDQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1557069978; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=PNgmQPvZ0m9b5E6+AqDfsuycGtwDzEFLFx5FPW6Pg5A=; b=o76fC1R3krUE5NA4Zp5dKTSZNmF2I/7SAnZOMZKZiVzYEk4apD/H5puv nuAfsZ+QSTFuO4ExD/oQyaZyToX6TYf+vLbQAztGcVgcjtiABzMW4wwNJ3 jGYe36iI6t1HeGtQ8Sflu5xhCxied3ZdkOee75gbTKvDgFjE579EEBW9ko P+4uRG4KPw43OsI7nFRs7zQ5R5y6O8xdJYXQZ7kqsCh9K75wUaZIli6j0h X1HdB7RwOJkeVI7FtotcOJFOx5i03NAisVmcPEp4ArP5q675c8Qq2h1kW0 qGW/n9LuywZJL0x3y2QCIOkgriPhdETv4UQJp04lwirmdC2jbo/cLA== Received: from fifthhorseman.net (ool-6c3a0662.static.optonline.net [108.58.6.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id BEA06F99D; Sun, 5 May 2019 11:26:18 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id 302C7204BB; Sun, 5 May 2019 11:22:51 -0400 (EDT) From: Daniel Kahn Gillmor To: David Bremner , "Rollins\, Jameson" , Notmuch Mail Subject: Re: parallelize test suite In-Reply-To: <878svlhlwc.fsf@tethera.net> References: <20190504205738.21504-1-jrollins@caltech.edu> <87r29dnbl0.fsf@fifthhorseman.net> <878svlhlwc.fsf@tethera.net> Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Sun, 05 May 2019 11:22:50 -0400 Message-ID: <87lfzllyd1.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 May 2019 15:26:20 -0000 --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Sat 2019-05-04 19:53:23 -0300, David Bremner wrote: > Last time we discussed parallel test running, there we concerns about > multiple versions of certain servers colliding with each other. This > still seems to be at least a theoretical issue with smtp-dummy, although > a glance suggests that it might only currently be used in T310-emacs.sh. > I'm not sure what a robust solution is here.=20 > > - gpg-agent > - emacs > - dtach > > Did I miss any other background processes run by the test suite? > > I can imagine gpg-agent is managed OK these days since it's started > automagically by gpg. gpg-agent is fine, because it's isolated by $GNUPGHOME, and each test uses a distinct $GNUPGHOME (see GNUPGHOME=3D"${TEST_TMPDIR}/gnupg in test-lib.sh) > emacs seems to use the current process id in the socket name, so that > also should be OK, although it should maybe be replaced with something > more robust to avoid problems with pid rollover. I _think_ including the > test name in the emacs server would do the trick I would have no objection to this improvement in isolation of the emacs server processes, but i think the pid rollover race condition is so minor that i don't think it sholud be a blocker for the adoption of this series. > The dtach socket is in the tmp.T* directory, so that should be OK. > > I wonder if a good solution would be to make running the test suite in > parallel be opt-in (e.g. by configuration option). Or at least have a > way to disable it for situations like CI and autobuilders. I agree that making it possible to force serialized tests would be good. I'd prefer that running tests in parallel be the default, though i wouldn't object to a ./configure --serialize-tests option if someone wants to implement it. I actually think that CI and autobuilders *should* exercise the parallel tests, as annoying as that might be initially, because it seems likely to catch any other potential entanglements. thanks for the review! --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXM7/ygAKCRB2GBllKa5f +BZJAQCdX8hzHOe8NBBYPHE6Zfzhs2fl83fhr11p8LcZ7WkU0AD/SyLt5Ts3iXsq 10JiOilJ01Mmc3E7p7WpGVkk15A+GgE= =Og5e -----END PGP SIGNATURE----- --=-=-=--