Re: GC thread performance

[Marko Rauhamaa <marko@pacujo.net>]

Linas Vepstas <linasvepstas@gmail.com>:
> I cannot speak to GC, but I freuqently encounter situations in guile
> where using the parallel constructs e.g. par-for-each, end up running
> slower than the single-threaded version. For example, using 2 or 3
> threads, I get a 2x and 3x speedup, great, but using 4x gives a
> slowdown, often 10x slower than single-threaded. I try to make sure
> that the insides of the loop are large and long-running, so that the
> cost of creating and scheduling threads is inconsequential.
>
> I have not attempted to determine the cause of this, but basically,
> that entire subsystem needs a careful review and measurement.

I'll have to speculate, too.

Guile guarantees the consistency of the data model under all
circumstances. Bad synchronization between threads is allowed to cause
unspecified behavior, but it should never trigger a SIGSEGV. In
practice, that means excessive locking: all data access needs to take
place in a critical section.

Linux (at least) has locking primitives that have virtually no overhead
when there is no contention. However, when there is contention, the
context is switched to the kernel and a memory barrier causes the CPU
hardware to flush its memory cache.

The same issue hampers C developers as well, although the newest C
standards explicitly shift to the coder the responsibility for the
consistency of the data model. Thus, in principle, a C programmer can
try clever tricks to eke out performance with multithreading.

CPython has famously grappled with the same performance issues. The
so-called GIL effectively turns every Python program into a
single-threaded process. The point of multithreading is not a gain in
performance. Rather, it is a programming paradigm to multiplex events.

These and other issues with multithreading have caused many of us to
reject multithreading as a paradigm altogether. Use event-driven
programming for multiplexing and multiprocessing for performance.


Marko