Uploaded image for project: 'Mender'
  1. Mender
  2. MEN-5794

C++ unit test system




      Acceptance criteria:

      • Select a test framework suitable for C++ unit tests.
      • For ease of use, it must be C++ based, not C.
      • Must support code coverage, at least indirectly (by compiler flags, etc).
      • Any Open Source license should be fine for this.
      • Set up a functioning dummy test (coordinate with MEN-5795 owner for build system to use; if not ready, then just use minimal Makefile).
      • Must pass in CI

      Client feature branch: feature-c++-client

      Selected test framework

      Selected by: Kristian Amlie

      There are a lot of test frameworks out there, and I definitely did not go through all of them.

      Shortly after I started, I discovered that the Google Test Framework has support for so-called "Death tests", which can use crashes and signal kills as part of the test flow and test conditions. Because this is used extensively in the unit tests for the Golang client (the state test which tests a lot of spontaneous reboot recovery situations, here and here), and the implementation was difficult and pretty horrible, I decided to make this a requirement for the final test framework.

      I used Wikipedia to comb through which frameworks supported this, which is far from foolproof, but nevertheless ruled out most frameworks, and in the end I was left with three: libcester, Criterion, and googletest.

      Libcester was eliminated early on. There was nothing wrong with it really, but it was clearly more simplistic than the other two, and offered less conveniences.

      I found both Criterion and googletest to be very good, and it was a close race between them. In the end I chose Googletest mainly because I found their documentation to be a bit better, and their death test framework has two different engines depending on what suits your test. Apart from that the two frameworks really only differed in secondary features, such as exactly how data driven tests are done, or how test logging is done. Either one was just fine in these respects.

      Googletest has an interesting approach to logging: Instead of providing the usual Log(DEBUG, "message") calls, you mark each scope (such as stack frame, or if block, or for loop) with a message, and these messages get printed if, and only if, your test fails, in a "backtrace-like" fashion. Unsure if this will be useful or just annoying, but if the latter, then logging with printf also works.


          Issue Links



              a10040 Kristian Amlie
              a10040 Kristian Amlie
              0 Vote for this issue
              1 Start watching this issue