Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-2524

Performance issues in policy loading



    • Type: Story
    • Status: Open
    • Priority: Medium
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:


      We noticed performance issues with ncf/Rudder policy loading, and especially with 3.9/3.10 (I attached a graph).

      These are the execution time of cf-agent -K on ncf base promises, without default promises.cf policy, and most of the time is spent running CFEngine code. I also ran these with callgrind and analyzed the results in kcachegrind.

      The difference between 3.9 and 3.10 seems to be mainly due to the new call to BundleResolvePromiseType in BundleResolve (vars are now parsed before and after classes). This call looks very expensive and increase the overall number of calls to malloc of 40%.

      PR: https://github.com/cfengine/core/pull/2753, which seems to get to the same duration/cost than 3.8.

      The duration augmentation from 3.8 to 3.9 seems to be caused by the move from RBTree to HashMaps (higher creation/deletion cost).

      More generally, policy loading is slow (always more than one second, for around 500 bundles), and consumes a lot of CPU resources, which becomes a problem in some environments. More than half of CPU cycles are apparently spent doing EvalContextStackPushPromiseFrame/EvalContextStackPopFrame in ExpandPromise, more specifically VariableTable creation/deletion in VariableTableCopyLocalized. This seems to copy all accessible variables, for each promise, to a new promise stack frame. I may not fully understand this, but I have the feeling it is not necessary to copy those variables, and that we could only lookup them where they are, do you know the reason for this?




            • Assignee:
              a10003 Eystein Maloy Stenberg
              amousset Alexis Mousset
            • Votes:
              0 Vote for this issue
              3 Start watching this issue


              • Created:

                Summary Panel