On a heavily loaded hub, system CPU usage is extremely high and cf-serverd has trouble serving all the requests. After a lot of debugging with @strace@, @gdb@ and @perf@, the conclusion is that the kernel wastes a lot of time on spinlocks, because of two pthread mutexes with extremely high contention:
- @malloc()@ mutex
- @OpenSSL@ internal error mutex
The @malloc()@ contention is much easier to debug and fix, and this might alleviate the high error rate (timeouts) that contends the second lock. So this bug is about fixing the malloc contention.