Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-3034

Improve code quality 2019 - valgrind, ASAN, LGTM, AFL



    • Type: Epic
    • Status: Done
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Epic Name:


      We have a relatively large C codebase. C is known for being unsafe in a number of ways. We are also seeing a lot of instability/problems, especially around parsing, database interactions, reporting etc. I think one of the best ways to combat this is to use state-of-the-art tools to find potentially dangerous bugs. This ticket is a collection of work done to improve our codebase and CI in this regard. The work started after I attended Black Hat 2018.

      LGTM - code analysis:

      • Builds / parses all our source code into an abstract syntax tree. Runs SQL-like queries on code to find known bad patterns.
      • Can be extended with custom queries - fix a bug and make sure it is never reintroduced
      • Partially open source
      • I picked this because:
        • Very accessible (they built cfengine core on the first try)
        • Well integrated with GitHub

      AddressSanitizer (ASAN) - Instrumentation to detect memory errors:

      • gcc / clang flag to add instrumentation and a runtime to C binaries
        • Provides a replacement for free / malloc etc
        • Adds shadow bytes around variables on stack and heap
      • detects bad memory usage in C binaries - use after free, buffer overflow, etc.
      • works at runtime, finds similar bugs like valgrind but is much more performant
      • Detects issues in both stack and heap memory
      • I picked this because:
        • Mentioned in talks by facebook and google as being the best/must have tool for big C/++ projects
      • We should add ASAN to unit tests and acceptance tests
        • Getting it working on all platforms might be too hard, but travis only would be very valuable also

      Valgrind - Runtime library to detect memory leaks:

      • Some of the same detection as ASAN
      • Easier to use, works on normal binaries (no build time options)
      • Much slower
      • Useful to add to a quick "bootstrap" / "deployment" test to see that our builds don't have easily detectable problems
      • Too slow for acceptance tests - use ASAN instead
      • I picked this because:
        • Easy to use and what we already recommend for users to find memory leaks

      American Fuzzy Lop (AFL) - Fuzz testing to find segfaults / programming errors:

      • Generate random test cases based on working inputs
      • I picked this because:
        • Praised at DEFCON / Blackhat 2018
        • Widely used by attackers / security researchers
        • Most popular fuzzer, easiest to get started and find documentation

      Success criteria:

      • Code is more stable, readable, maintainable and less buggy
      • Bugs are discovered in PRs before merge (Travis + LGTM)


          Issue Links



              • Assignee:
                olehermanse Ole Herman Schumacher Elgesem
                olehermanse Ole Herman Schumacher Elgesem
              • Votes:
                0 Vote for this issue
                1 Start watching this issue


                • Created:

                  Summary Panel