Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-1329

New logging framework to determine log destination and filtering

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Open
    • Priority: Low
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Logging
    • Labels:
      None

      Description

      Please note that this is a early specification that is likely to change according to feedback and feasibility validation.

      The purpose of this specification is to create a generic logging framework that either 1) solves the relevant issues/subtasks or 2) enables solving them in the future with minor additions.
      This specification only determines if an entry is logged and to which destinations, it does not change the format of the message output (this is an orthogonal issue).

      Description

      Note that this description is inspired by "syslog-ng configuration":https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/syslog-ng.conf.5.html because many CFEngine users are familiar with this model and it is proven.

      Log entry

      This is the "raw" log entry, consisting of several attributes:

      • timestamp; all have this
      • message (the actual text); all have this
      • level (error, inform, verbose, debug); all have this
      • promise_type (that caused the log); only if this is a message generated by a promise (i.e. removing the promise would remove the message)
      • promise_outcome (that caused the log); only if this is a message about a promise being kept/repaired/not kept
      • (might be some more in the future)

      Sources

      Sources decide what the input to the filter will be, for example specific promise types, or non-policy-related log entries.

      Filters

      Filters take all log entries and either output or filter them, based on the policy they have set.
      For example, a filter that only output entries with level error or higher (@level_min => "error"@), or one that only outputs messages about not kept promises (@promise_outcome => "notkept"@). They can also make more advanced expressions, but the point is that they decide if the log entry passes through or not based on the attributes.

      Log destinations

      These options decide where and how things get logged, e.g. to syslog, console, and potentially how they are logged. For example the syslog facility (ref #3041).

      Log policy selection

      The usual "most specific match" approach is taken to decide which log policy is in effect for a given log entry.

      1. If the entry is generated by a promise and there is a log policy attribute on the promise, this policy is taken.
      2. If there is an entry on the component (e.g. @body agent control@, @body server control@), this is taken.
      3. If there is no log policy on either, the global log policy is taken, i.e. either from @body common control@ or default programmed setting.

      Syntax

      Common log options

      <pre>
      body common control # can also be body component control, e.g. body agent control
      {
      log =>

      { <name1>, ... }

      ;
      }
      </pre>

      Note that handling duplicates is the responsibility of the user (e.g. of <name1> appears twice in the list all messages will be duplicated).

      <pre>
      body log <name1>
      {
      log_source => <name2>; # Only makes sense in common control or component control, if omitted defaults to all.
      log_filter => <name3>;
      log_destination => <name4>;
      }
      </pre>

      Sources

      <pre>
      body log_source <name2>
      {
      promise_type =>

      { "commands" };
      promise_messages_only => "false"; # policy related messages only?, defaults to false to get all log messages
      }
      </pre>

      h3. Filters

      For language consistency and flexibility, these follow the same approach as "file_select":https://docs.cfengine.com/lts/reference-promise-types-files.html#file_select.

      <pre>
      body log_filter <name3>
      {
      level_min => "error/inform/verbose/debug";
      level => "error/inform/verbose/debug";
      message_regex => "...";
      promise_outcome => { ... };
      log_result => "<boolean expression on log_filter attributes>";
      }
      </pre>

      If log_result evaluates to @true@, the message is logged, otherwise it is filtered.

      h3. Destinations

      <pre>
      body log_destination <name4>
      {
      log_destination => { "console", "syslog", "file" };
      file_log_path => "/tmp/logfile.log"; # Only relevant if file is specified as log_target
      syslog_facility => "local1"; # Only relevant if syslog is specified as log_target
      console_color => "red"; # Only relevant if console is specified as log_target and term is color enabled
      }
      </pre>

      h3. Log policy selection

      The log policy is decided by having optional (global) log attributes and pick the first found, in this order:
      # promise attribute
      # @body <component> control@ (e.g. body agent control)
      # @body common control@

      For example:

      <pre>
      body common control
      {
      log => { default };
      }

      body server control
      {
      log => { debug };
      }

      bundle agent noisy
      {
      commands:
      "/bin/noisy"
      log => { silent };
      }

      body log default {...}
      body log debug {...}
      body log silent {...}
      </pre>

      h2. Use case: Completely silence a specific promise

      <pre>
      bundle agent noisy
      {
      commands:
      "/bin/noisy"
      log => { silent };
      }

      body log silent
      {
      log_filter => silent;
      }

      body log_filter silent
      {
      log_result => ""; # never matches anything
      }
      </pre>

      h2. Use case: Globally disable syslog output, otherwise behave as default

      <pre>
      body common control
      {
      log => { console_only };
      }

      body log console_only
      {
      log_destination => console_only;
      }

      body log_destination console_only
      {
      log_destination => { "console" };
      }
      </pre>

      h2. Use case: Never log not kept messages from agent commands promises to keep backward-compatibility with CFEngine 3.3

      <pre>
      body agent control
      {
      log => { commands_notkept_silent };
      }

      bundle agent command
      {
      commands:
      "/bin/true";
      "/bin/false"; # would normally log error due to nonzero return code, but silent with this agent log policy
      }

      body log commands_notkept_silent
      {
      log_source => commands;
      log_filter => notkept_silent;
      }

      body log_source commands
      {
      promise_type => { "commands" }

      ;
      }

      body log_filter notkept_silent
      {
      promise_outcome =>

      { "notkept" }

      ;
      log_result => "!(promise_outcome.promise_type)";
      }
      </pre>

      Use case: Never log output from vars promises to silence error messages about files not found in read* functions, otherwise behave as default

      <pre>
      body common control
      {
      log =>

      { vars_silent }

      ;
      }

      body log vars_silent
      {
      log_source => vars_silent;
      }

      body log_filter vars_silent # excludes vars promises
      {
      promise_type =>

      { "commands", "files", ... }

      ;
      }
      </pre>

      Use case: Log all errors to a file for ticketing system integration, the rest to console and syslog

      <pre>
      body common control
      {
      log =>

      { file_error, default }

      ; # assuming default works as 3.8 logging (should be defined in lib/ somewhere)
      }

      body log file_error
      {
      log_filter => level("error");
      log_destination => file("/tmp/cfengine_error.log");
      }

      body log_filter level(level)
      {
      level => "$(level)";
      log_result => "level";
      }

      body log_destination file(path)
      {
      log_destination =>

      { "file" };
      file_log_path => "$(path)";
      }
      </pre>


      h2. Use case: Log cf-serverd debug (and above) output to a file to catch an intermittent connection error, otherwise behave as default

      <pre>
      body server control
      {
      log => { file_debug, default }; # assuming default works as 3.8 logging (should be defined in lib/ somewhere)
      }

      body log file_debug
      {
      log_filter => level_min("debug");
      log_destination => file("/tmp/cf-serverd-debug.log");
      }

      body log_filter level_min(level)
      {
      level_min => "$(level)";
      log_result => "level_min";
      }

      body log_destination file(path)
      {
      log_destination => { "file" }

      ;
      file_log_path => "$(path)";
      }

      </pre>

      TODO / additional requirements

      • It must be possible to silence some specific read* functions AND notkept commands. (Perhaps let *_log_filter be a list, but that gets pretty complex and can lead to duplicate logs if a message passes through both filters?)
      • It should be possible to silence specific functions (e.g. silence readfile() but not grep()); perhaps use stackpath (ala leaf_name, path_name in file_select)
      • Feasibility estimate; can this realistically be achieved? In particular, is this feasible:
        • the body-in-body approach in log.log_filter.attribute.
        • body <component> control overrides body common control.
      • Handle component CLI options (e.g. -v, -I, etc.) of log level to avoid surprise of missing logs due to policy filtering, especially for default. E.g. console should get all verbose level in -v, but what if console_log_filter is set to level_min => error? Probably the component options (e.g. -v) overrides everything for console output, i.e. it has the highest priority (higher than the promise attribute), or at least nothing can have *lower* output level than the console option.
      • Which promise type is a function (e.g. if it's wrapped in ifvarclass()). Intuitively it is the promise type of the return value type (vars or classes).
      • Migration/deprecation plan for existing log options
      • Figure out how to represent defaults (to be compatible with current versions)
      • Figure out minimal set of log entry attributes
      • It would be nice to have an additional (regex) filter on messages to be emailed. Probably a special filter in body executor control. (see #7938)
      • Should we support logging to specific file? Should the file be hardcoded or selected by user? Can it grow indefinitely?
      • Consider merging log_source and log_filter, as the use case of filtering not kept commands promises does not seem to be possible if they are split.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                a10003 Eystein Maloy Stenberg
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel