Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-2159

High sys CPU usage because of malloc() lock contention on heavily loaded cf-serverd

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.7
    • Component/s: cf-serverd
    • Labels:
      None

      Description

      On a heavily loaded hub, system CPU usage is extremely high and cf-serverd has trouble serving all the requests. After a lot of debugging with @strace@, @gdb@ and @perf@, the conclusion is that the kernel wastes a lot of time on spinlocks, because of two pthread mutexes with extremely high contention:

      1. @malloc()@ mutex
      2. @OpenSSL@ internal error mutex

      The @malloc()@ contention is much easier to debug and fix, and this might alleviate the high error rate (timeouts) that contends the second lock. So this bug is about fixing the malloc contention.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                a10038 jimis (Dimitrios Apostolou)
                Reporter:
                a10038 jimis (Dimitrios Apostolou)
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel

                    Time Tracking

                    Estimated:
                    Original Estimate - 3 hours
                    3h
                    Remaining:
                    Not Specified
                    Logged:
                    Time Not Required
                    Not Specified