Re: parallelize test suite

[David Bremner <david@tethera.net>]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> On Sat 2019-05-04 20:57:43 +0000, Rollins, Jameson wrote:
>> This is a simple patch series that will run the entire test suite in
>> parallel if either the moreutils or GNU parallel utility is
>> available.  On my 8-core machine the full test suite will now run in
>> under 20 seconds, which is a pretty huge improvement.
>
> I've reviewed this series, and it is a *very* nice cleanup; the latency
> reduction makes it much more pleasant to develop notmuch safely.
>
> My computer is weaker than jamie's (i have an older 4-core i5
> processor), but with this series applied, the test suite went from 1m48s
> to 0m29s for me.
>

Last time we discussed parallel test running, there we concerns about
multiple versions of certain servers colliding with each other. This
still seems to be at least a theoretical issue with smtp-dummy, although
a glance suggests that it might only currently be used in T310-emacs.sh.
I'm not sure what a robust solution is here. 

- gpg-agent
- emacs
- dtach

Did I miss any other background processes run by the test suite?

I can imagine gpg-agent is managed OK these days since it's started
automagically by gpg.

emacs seems to use the current process id in the socket name, so that
also should be OK, although it should maybe be replaced with something
more robust to avoid problems with pid rollover. I _think_ including the
test name in the emacs server would do the trick

The dtach socket is in the tmp.T* directory, so that should be OK.

I wonder if a good solution would be to make running the test suite in
parallel be opt-in (e.g. by configuration option). Or at least have a
way to disable it for situations like CI and autobuilders.

d