Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-3406

Allow url_get() to return content key as json instead of string



    • Type: Story
    • Status: Open
    • Priority: (None)
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: function: url_get()
    • Labels:


      It would be nice if I did not have to explicitly convert content in a response from url_get() to a data container.

      Currently, (as of 3.16.0) if I am working with a JSON API, I have to pull the content key into it's own data container as it's simply a string in the response.

      For example:

      Example illustrating need to pull content key into own data container
      bundle agent example_url_get_wtfismyip_v0
            # This is the URL that we will query.
            "url" string => "https://wtfismyip.com/json";
            # We don't need to supply any specific options in this case.
            # url.max_content is a typical setting to change. 4K isn't enough for
            # some, (many?) API responses.
            "options" data => '{}';
            # Here we make our http(s) call, storing the response in `d`
            "d" data => url_get( $(url), @(options));
            # The response content is returned as a *string* inside the `content`. In
            # order to work with it, we need to pull it into a proper data container.
            "content" data => "$(d[content])";
            # Let's view the multi-line JSON representation of the `url_get()` response.
            "url_get() result:$(with)" with => string_mustache( "{{%-top-}}", d);
            # Let's view the multi-line JSON representation of our extracted `content`
            # datacontainer.
            "Extracted content as data container: $(with)"
              with => string_mustache( "{{%-top-}}", content);
      bundle agent __main__{methods:"example_url_get_wtfismyip_v0";}
      R: url_get() result:{
        "content": "{\n    \"YourFuckingIPAddress\": \"\",\n    \"YourFuckingLocation\": \"Lawrence, KS, United States\",\n    \"YourFuckingHostname\": \"24-143-37-227-dynamic.midco.net\",\n    \"YourFuckingISP\": \"Midco\",\n    \"YourFuckingTorExit\": false,\n    \"YourFuckingCountryCode\": \"US\"\n}\n",
        "headers": "HTTP/1.1 200 OK\r\nAccess-Control-Allow-Methods: GET\r\nAccess-Control-Allow-Origin: *\r\nContent-Type: application/json\r\nDate: Fri, 28 Aug 2020 14:25:59 GMT\r\nContent-Length: 268\r\n\r\n",
        "rc": 0,
        "returncode": 200,
        "success": true
      R: Extracted content as data container: {
        "YourFuckingCountryCode": "US",
        "YourFuckingHostname": "24-143-37-227-dynamic.midco.net",
        "YourFuckingIPAddress": "",
        "YourFuckingISP": "Midco",
        "YourFuckingLocation": "Lawrence, KS, United States",
        "YourFuckingTorExit": false

      Perhaps if the response headers included Content-Type: application/vnd.api+json the content key could be treated as a data container from the start, making it un-necessary to have an additional variable.


      Or, perhaps an optional argument to url_get() could cause it.




            • Assignee:
              a10042 Nick Anderson
            • Votes:
              0 Vote for this issue
              1 Start watching this issue


              • Created:

                Summary Panel