Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-2084

implement def.json parsing in C with multiple data files

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.12.0
    • Component/s: Evaluation
    • Labels:
      None

      Description

      The augments file (def.json) facility has been very popular with new users and this proposal extends it to be more friendly with multiple external data sources.

      Pull Request: https://github.com/cfengine/core/pull/2515

      Really simple logic currently:

      • augments key contents (which must be an array of strings) are loaded.
      • System variables like sys.inputdir are expanded (see acceptance test).
      • vars and classes are merged because they are processed one by one
      • inputs override other inputs

      Merge strategies

      It would be nice if each additional augments could be defined with its own merge strategy.

      An entry could be of the form

      { "file": "myfile.json", "merge": "deep|overwrite|function" }
      

      Allowing the following where sys.flavor.json is merged with the default strategy and the host-specific augments is merged deeply to extend the data instead of overwriting it.

      {
       "augments": [ 
          "$(sys.workdir)/cmdb/$(sys.flavor).json",
          { "file": "$(sys.workdir)/cmdb/$(sys.fqhost).json": "merge": "deep" }
          { "file": "$(sys.workdir)/cmdb/$(sys.key_digest).json": "merge": "FUNCTION()" }
        ]
      }
      

      Current override strategy used by mergedata()

      Here is an example of the extension so that after loading def.json we merge it with a flavor specific augments, followed by a host-specific augments.

      {
       "augments": [ "$(sys.workdir)/cmdb/$(sys.flavor).json", "$(sys.workdir)/cmdb/$(sys.fqhost).json" ]
      }
      

      Here is an example showing how vars would be merged using the above augments specification on a Centos 6 host.

      sys.flavor.json (centos_6.json)

      {
       "vars": {
          "dir_templates": "$(sys.workdir)/templates/$(sys.flavor).json",
          "setting_2": "value for setting 2",
          "deep": {
            "keys": "are not merged deeply"
          }
        }
      }
      

      sys.fqhost.json (host001.json)

      {
       "vars": {
          "owner": "nick anderson",
          "setting_2": "Setting 2 has been overridden at the host level"
          "deep": {
            "overrides": "Last object wins"
        }
      }
      

      Expected merged result after expansion:

      {
       "vars": {
              "dir_templates": "/var/cfengine/templates/centos_6.json",
              "owner": "nick anderson",
              "setting_2": "Setting 2 has been overridden at the host level",
              "deep": {
                "overrides": "Last object wins"
             }
        }
      }
      

      And here are the expected variable values:

      def.dir_templates = /var/cfengine/templates/centos_6.json
      def.owner = nick anderson
      def.setting_2 = Setting 2 has been overridden at the host level
      def.deep = { "overrides": "Last object wins" };
      

      Deep merge

        Attachments

        1. def.json
          0.2 kB
        2. promises.cf
          0.8 kB

          Issue Links

            Activity

              People

              • Assignee:
                a10038 jimis (Dimitrios Apostolou)
                Reporter:
                tzz Ted Zlatanov
              • Votes:
                4 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: