CxxTest is different from JUnit

I’ve been stretching my TDD practice recently by dipping into the murky world of C++ again after a break of a few years. After reading a number of comparisons – notably this one by Noel Llopis – we settled on CxxTest as the project’s TDD framework. I then spent a few days converting and refactoring a few existing tests, and writing a load of new ones, to get an initial test suite going.

I discovered one big surprise along the way: unlike JUnit et al, each test suite class is only instantiated once, regardless of how many testXXX() methods it has. This means that the suite’s setUp() and tearDown() methods are always acting on the same fixture instance throughout the test run; and the test suite’s constructor is only invoked once. So for test independence it’s essential to only put pointers into the test’s state, so that all test objects can be safely deleted.

Another minor irritation is that CxxTest doesn’t easily allow me to run a subset of the tests. This is a useful feature when working with legacy code, because it’s sometimes hard to locate a crash among a few hundred tests.

What else should I know about? Are there any other gotchas? Have you found other styles of working with CxxTest?

(Note: CxxTest is also commonly, and mistakenly, known as CxxUnit.)

Advertisements

10 thoughts on “CxxTest is different from JUnit

  1. Hi! We’ve been using CxxTest for quite a while, and are pretty happy with it. We have integrated cxxtest as a post-build step in Visual Studio. When you use
    –runner=ParenPrinter you can doubleclick an error message to jump directly to the problematic sourcecode. The build fails if the tests are not run successful.

    The post-build event calls nmake with a Makefile, that contains a variable TESTS which defaults to Test*.h, so that all Tests are included. In case of a bug I modify the Makefile to include just the problematic file so that tests are run more quickly.

  2. It’s too late for you now, I suppose, but I will immodestly suggest my framework, at <http://unittest.red-bean.com/&gt;. Its main distinction from other C++ test frameworks out there is that it doesn’t have setUp and tearDown functions: it uses the test class’s constructor and destructor instead. Also (because of C++’s lack of reflection) it represents tests by classes instead of by member functions. So if you want a bunch of tests to all, say, set themselves up and tear themselves down in the same way, just have them inherit from the same base class. (And individual tests can tweak that further by sticking more stuff in their constructors/destructors.)

    We’ve been using it for a little more than three years at work now; it does the job for us. And it avoids the weirdnesses that you mention. It does allow you to run a subset of the tests, too, though if you want to locate a crash, it installs a signal handler that will tell you the test that was running at the time.

    (You don’t allow much in the way of HTML in comments here, do you? No links, no paragraphs…)

  3. Hi Martin, I use –runner=ParenPrinter too, with the tests set up as an external tool. So we have to run them manually after each build. I like the idea of the nmake job as a post-build step – as you say, it moves the test suite configuration into an editable file, instead of being hidden in the project properties as I have it now. Actually I have the tests stored in a separate project which builds only if the product’s build succeeded – I’m sure I can extend it to run the tests too. Thanks for the tip!

  4. Hi David, I’ll certainly take a look at red-bean – thanks for the link!

    (I receive so many spam comments that I’ve been close to disabling them altogether. I’ll re-enable HTML now and see what happens…)

  5. Hi Kevin,

    I was wondering if you had to modify the the includes in the .h and .cpp files in the cxxtest directory. I had to change them from:

    #include to #include “blah.h”

    for files such as Descriptions.h, Descriptions.cpp, etc.

    since the folder hierarchy is flat i’m not sure why the cxxtest folder prefix is included in the #include statement. all of the header and source files are in the same directory.

    any help would be appreciated.

    fyi: using visual studio 2005 32-bit compiler version 12.00.8168 80×86

    thanks.

  6. We’ve recently incorporated CxxTest in a new project, and I’m quite happy with it, however, I would like to break down the tests into more than one suite rather than a single monolithic header and generated file.

    I’ve tried to break up the suites by category and generate the relevant source with the –part option, but each generated file contains a main, causing link errors ( I follow this up with a main file and a –root generation option with no input file).

    Have you or anyone else been successful with multiple suites in a single test executable?

    Thanks in advance,
    Dennis

  7. Hi Dennis,
    At present we do this only rarely, eg. when there’s a memory leak somewhere in the tests so that we need to isolate the test that causes the problem. In such cases we take the brute-force approach and simply build fewer test suites, usually by changing our MSVC build event scripts. Not ideal, but just about acceptable in our case.
    Cheers, Kevin

  8. Hi Dennis,

    I’ve been using CxxText for a few years now. There is no problem separating your tests into a number of files. The simple way is just to specify all your test files to cxxtestgen. The “good” way though, is as you specified, to use cxxtestgen with –part and then create the main with cxxtestgen –root. If you have any issues with that I’ll be more than happy to help.

    Ron.

  9. Thanks Kevin. I also found CxxUnit via Noel’s excellent comparison having dabbled with CppUnit but not enjoyed its verbosity.

    Dennis – or anyone attempting to separate tests into several source files to combine into a single runner – main() is only generated if you run cxxtestgen with –runner or –gui

    My makefile looks like this:
    run_tests: runner
    ./runner

    runner: runner.cpp ${tests} ${objects}
    ${CXX} $^ -o $@

    runner.cpp:
    cxxtestgen.pl -o $@ –error-printer –root

    ${tests}: %.cpp: %.h
    cxxtestgen.pl -o $@ –part $^

    Hope that helps anyone who puzzled for a while like me :-)

    –rob

  10. @Rob

    Thanks, I was looking for just this remark! I already started to design a setup where a number of exec would be created with a number of scripts tying everything back together, but this is the solution ( and a simple one at it too.. )

    kr,
    – Dirk

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s