Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-2084

implement def.json parsing in C with multiple data files



    • Type: Task
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.12.0
    • Component/s: Evaluation
    • Labels:


      The augments file (def.json) facility has been very popular with new users and this proposal extends it to be more friendly with multiple external data sources.

      Pull Request: https://github.com/cfengine/core/pull/2515

      Really simple logic currently:

      • augments key contents (which must be an array of strings) are loaded.
      • System variables like sys.inputdir are expanded (see acceptance test).
      • vars and classes are merged because they are processed one by one
      • inputs override other inputs

      Merge strategies

      It would be nice if each additional augments could be defined with its own merge strategy.

      An entry could be of the form

      { "file": "myfile.json", "merge": "deep|overwrite|function" }

      Allowing the following where sys.flavor.json is merged with the default strategy and the host-specific augments is merged deeply to extend the data instead of overwriting it.

       "augments": [ 
          { "file": "$(sys.workdir)/cmdb/$(sys.fqhost).json": "merge": "deep" }
          { "file": "$(sys.workdir)/cmdb/$(sys.key_digest).json": "merge": "FUNCTION()" }

      Current override strategy used by mergedata()

      Here is an example of the extension so that after loading def.json we merge it with a flavor specific augments, followed by a host-specific augments.

       "augments": [ "$(sys.workdir)/cmdb/$(sys.flavor).json", "$(sys.workdir)/cmdb/$(sys.fqhost).json" ]

      Here is an example showing how vars would be merged using the above augments specification on a Centos 6 host.

      sys.flavor.json (centos_6.json)

       "vars": {
          "dir_templates": "$(sys.workdir)/templates/$(sys.flavor).json",
          "setting_2": "value for setting 2",
          "deep": {
            "keys": "are not merged deeply"

      sys.fqhost.json (host001.json)

       "vars": {
          "owner": "nick anderson",
          "setting_2": "Setting 2 has been overridden at the host level"
          "deep": {
            "overrides": "Last object wins"

      Expected merged result after expansion:

       "vars": {
              "dir_templates": "/var/cfengine/templates/centos_6.json",
              "owner": "nick anderson",
              "setting_2": "Setting 2 has been overridden at the host level",
              "deep": {
                "overrides": "Last object wins"

      And here are the expected variable values:

      def.dir_templates = /var/cfengine/templates/centos_6.json
      def.owner = nick anderson
      def.setting_2 = Setting 2 has been overridden at the host level
      def.deep = { "overrides": "Last object wins" };

      Deep merge


        1. def.json
          0.2 kB
        2. promises.cf
          0.8 kB

          Issue Links



              a10038 jimis (Dimitrios Apostolou)
              tzz Ted Zlatanov
              4 Vote for this issue
              15 Start watching this issue