After reviewing the source for the four C++ unit testing frameworks with respect to if their output can be normalized without modifying the frameworks, I found that creating the normalized output solution for Google Test would require parsing the error strings formed at a higher level of the framework’s stack. In turn I decided to include CppUTest as a fifth framework, but it turned out that CppUTest actually had the same issue.
The code necessary to normalize the output for the 5 frameworks has been written and the points of interest regarding each framework follow. Next up is to evaluate the implementation of assertions in four of the frameworks to determine if any of the framework’s can be made thread safe during the the execution of a test just by providing a thread safe custom formatter. boost::unit_test has been evaluated already and it is necessary to modify that framework’s code for it to be the thread safe.
- Of the five frameworks, setting up the custom reporter for UnitTest++ was the quickest.
- It has a simple reporter and the design of the reporter aligned well with my desired output format and content.
TUT / Template Unit Test Framework
- Of the five frameworks, setting up the custom reporter for TUT was the second quickest.
- Like UnitTest++ its reporter was simple, but its design was not aligned as closely with my desired output format and content.
boost::unit_test / Boost Test Library
- Of the five frameworks, setting up the custom reporter for boost::unit_test was the most time consuming (by several factors).
- It has two different formatters involved in output, and it allows overriding the stream that each formatter uses.
- The current docs do not detail the use of custom formatters. Since the code is also more complex, it was necessary to spend more time reviewing framework code.
- Setting the formatter has to be done after other initialization or it will be overwritten by the framework.
- It was not necessary to parse the assertion failure string reducing the fragility of the custom output solution.
gtest / Google Test
- The custom printer solution has some fragility as it was necessary to parse the assertion failure strings to pull out the information of interest and disregard additional formatting added at a higher level in the framework’s stack.
- The custom output solution has some fragility as it was necessary to parse the assertion failure strings to pull out the information of interest and disregard additional formatting added at a higher level in the framework’s stack.