Intro stuff
The Standard C library
assert( ) macro is brief, to the point and portable. In addition,
when you’re finished debugging you can remove all the code by defining
NDEBUG, either on the command-line or in code.
Also, assert( ) can be used
while roughing out the code. Later, the calls to assert( ) that are
actually providing information to the end user can be replaced with more
civilized messages.
Testing is a necessary evil to many
programmers, but it doesn't have to be all that evil.
The title of this section is a variation
on a theme from Extreme
Programming[5],
or XP for short. XP is a code-centric discipline for getting software done
right, on time, within budget, while having fun along the way.
The XP approach is to take the best
software practices to the extreme. For example, if code reviews are good, then
code should be reviewed constantly, even as it's written. Hence the XP practice
of pair programming, where all code is written by two developers sharing
a single workstation. One programmer pilots the keyboard while the other watches
to catch mistakes and give strategic guidance. Then they switch roles as needed.
The next day they may pair up with other folks.
Likewise, if testing is good, then all
tests should be automated and run many times per day. An ever growing suite of
unit tests should be executed whenever you create or modify any function, to
ensure that the system is still stable. Furthermore, developers should integrate
code into the complete, evolving system and run functional tests often (at least
daily).
You've probably seen the old cartoon with
the caption, "You guys start coding while I go find out what they want." I spent
a number of years as a developer wondering why users couldn't figure out what
they wanted before I started coding. I found it very frustrating to attend a
weekly status meeting only to discover that what I completed the week before
wasn't quite going to fit the bill because the analysts changed their mind. It's
hard to reshape concrete while it's drying. Only once in my career have I had
the luxury of a "finished spec" to code from.
Over the years, however, I've discovered
that it is unreasonable to expect mere humans to be able to articulate software
requirements in detail without sampling an evolving, working system. It's much
better to specify a little, design a little, code a little, test a little. Then,
after evaluating the outcome, do it all over again. The ability to develop from
soup to nuts in such an iterative fashion is one of the great advances of this
object-oriented era in software history. But it requires nimble programmers who
can craft resilient (i.e., slow-drying) code. Change is hard.
Ironically, another kind of change that
good programmers want desperately to perform has always been opposed by
management: improving the physical design of working code. What maintenance
programmer hasn't had occasion to curse the aging, flagship company product as a
convoluted patchwork of spaghetti, wholly resistant to modification? The fear of
tampering with a functioning system, while not totally unfounded, robs code of
the resilience it needs to endure. "If it ain't broke, don't fix it" eventually
gives way to "We can't fix it — rewrite it." Change is
necessary.
Fortunately, we are now seeing the rise
of the new discipline of Refactoring, the art of internally restructuring
code to improve its design, without changing the functionality visible to the
user[6]. Such tweaks
include extracting a new function from another, or its inverse, combining
methods; replacing a method with an object; parameterizing a method or class; or
replacing conditionals with polymorphism. Now that this process of improving a
program's internal structure has a name and the support of industry luminaries,
we will likely be seeing more of it in the workplace.
But whether the force for change comes
from analysts or programmers, there is still the risk that changes today will
break what worked yesterday. What we're all after is a way to build code that
withstands the winds of change and actually improves over time.
Many practices purport to support this
quick-on-your-feet motif, of which XP is only one. In this section I explore
what I think is the key to making incremental development work: a ridiculously
easy-to-use automated unit test framework, which I have implemented not only in
C++, but also in C and Java (although the latter versions are not shown
here).
Unit tests are what developers write to
gain the confidence to say the two most important things that any developer can
say:
I can't think of
a better way ensure that you know what the code you're about to write should do
than to write the unit tests first. This simple exercise helps focus the mind on
the task ahead, and will likely lead to working code faster than just jumping
into coding. Or, to express it in XP terms, Testing + Programming is faster than
just Programming. Writing tests first also puts you on guard up front against
boundary conditions that might cause your code to break, so your code is more
robust right out of the chute.
Once you have code that passes all your
tests, you then have the peace of mind that if the system you contribute to
isn't working, it's not your fault. The statement, "All my tests pass" is a
powerful trump card in the workplace that cuts through any amount of politics
and hand waving.
Writing good unit tests is so important
that I'm amazed I didn't discover its value earlier in my career. Let me
rephrase that. I'm not really amazed, just disappointed. I still remember what
turned me off to formal testing at my first job right out of school. The testing
manager (yes, we had one in 1978!) asked me to write a unit-test plan, whatever
that was. Being an impatient youth I thought it was silly to waste time writing
a plan — why not just write the test? That encounter soured me on the idea
of formal test plans for years
thereafter.
I think that most developers, like
myself, would rather write code than write about code. But what does a unit test
look like? Quite often developers just verify that some well behaved input
produces the expected output, which they inspect visually. Two dangers exist in
this approach. First, programs don't always receive just well behaved input. We
all know that we should test the boundaries of program input, but it's hard to
think about it when you're trying to just get things working. If you write the
test for a function first before you start coding, you can wear your QA hat and
ask yourself, "What could possibly make this break?" Code up a test that will
prove the function you'll write isn't broken, then put on your developer hat and
make it happen. You'll write better code than if you hadn't written the test
first.
The second danger is inspecting output
visually to see if things work. It's fine for toy programs, but production
software is too complex for that kind of activity. It is tedious and error prone
to visually inspect program output to see if a test passed. Most any such thing
a human can do a computer can do, but without error. It's better to formulate
tests as collections of Boolean expressions and have the test program report any
failures.
As an example, suppose you need to build
a Date class in C++ that has the following properties:
Your class could
store three integers representing the year, month, and day. (Just be sure the
year is 16 bits or more to satisfy the last bullet above.) The interface for
your Date class might look like this:
//: C02:Date.h #ifndef DATE_H #define DATE_H #include <string> struct Duration { int years; int months; int days; Duration(int y, int m, int d) : years(y), months(m), days(d) {} }; class Date { public: Date(); Date(int year, int month, int day); Date(const std::string&); int getYear() const; int getMonth() const; int getDay() const; std::string toString() const; friend Duration duration(const Date&, const Date&); friend bool operator<(const Date&, const Date&); friend bool operator<=(const Date&, const Date&); friend bool operator>(const Date&, const Date&); friend bool operator>=(const Date&, const Date&); friend bool operator==(const Date&, const Date&); friend bool operator!=(const Date&, const Date&); private: int year; int month; int day; int compare(const Date&) const; static int daysInPrevMonth(int year,int mon); }; #endif ///:~
The implementation for this class looks
like this:
//: C02:Date.cpp {O} #include "Date.h" #include <string> #include <algorithm> // for swap() #include <ctime> #include <cassert> #include <sstream> #include <iomanip> using namespace std; namespace { const int daysInMonth[][13] = { {0,31,28,31,30,31,30,31,31,30,31,30,31}, {0,31,29,31,30,31,30,31,31,30,31,30,31}}; inline bool isleap(int y) { return y%4 == 0 && y%100 != 0 || y%400 == 0; } } Date::Date() { // Get current date time_t tval = time(0); struct tm *now = localtime(&tval); year = now->tm_year + 1900; month = now->tm_mon + 1; day = now->tm_mday; } Date::Date(int yr, int mon, int dy) { assert(1 <= mon && mon <= 12); assert(1 <= dy && dy <= daysInMonth[isleap(year)][mon]); year = yr; month = mon; day = dy; } Date::Date(const std::string& s) { // Assume YYYYMMDD format istringstream is(s); is >> setw(4) >> year; is >> setw(2) >> month; is >> setw(2) >> day; } int Date::getYear() const { return year; } int Date::getMonth() const { return month; } int Date::getDay() const { return day; } string Date::toString() const { ostringstream os; os << setw(4) << year << setw(2) << month << setw(2) << day; return os.str(); } int Date::compare(const Date& d2) const { int result = year - d2.year; if (result == 0) { result = month - d2.month; if (result == 0) result = day - d2.day; } return result; } int Date::daysInPrevMonth(int year, int month) { if (month == 1) { --year; month = 12; } else --month; return daysInMonth[isleap(year)][month]; } bool operator<(const Date& d1, const Date& d2) { return d1.compare(d2) < 0; } bool operator<=(const Date& d1, const Date& d2) { return d1.compare(d2) <= 0; } bool operator>(const Date& d1, const Date& d2) { return d1.compare(d2) >= 0; } bool operator>=(const Date& d1, const Date& d2) { return d1.compare(d2) >= 0; } bool operator==(const Date& d1, const Date& d2) { return d1.compare(d2) == 0; } bool operator!=(const Date& d1, const Date& d2) { return d1.compare(d2) != 0; } Duration duration(const Date& date1, const Date& date2) { int y1 = date1.year; int y2 = date2.year; int m1 = date1.month; int m2 = date2.month; int d1 = date1.day; int d2 = date2.day; // Compute the compare int order = date1.compare(date2); if (order == 0) return Duration(0,0,0); else if (order > 0) { // Make date1 precede date2 locally using std::swap; swap(y1, y2); swap(m1, m2); swap(d1, d2); } int years = y2 - y1; int months = m2 - m1; int days = d2 - d1; assert(years > 0 || years == 0 && months > 0 || years == 0 && months == 0 && days > 0); // Do the obvious corrections (must adjust days // before months!) - This is a loop in case the // previous month is February, and days < -28. int lastMonth = m2; int lastYear = y2; while (days < 0) { // Borrow from month assert(months > 0); days += Date::daysInPrevMonth( lastYear, lastMonth--); --months; } if (months < 0) { // Borrow from year assert(years > 0); months += 12; --years; } return Duration(years, months, days); } ///:~
You can now write tests for the functions
you want to implement first, something like the following:
//: C02:SimpleDateTest.cpp //{L} Date #include "Date.h" #include <iostream> using namespace std; int nPass = 0, nFail = 0; void test(bool t) { if(t) nPass++; else nFail++; } int main() { Date mybday(1951, 10, 1); test(mybday.getYear() == 1951); test(mybday.getMonth() == 10); test(mybday.getDay() == 1); cout << "Passed: " << nPass << ", Failed: " << nFail << endl; } /* Output: Passed: 3, Failed: 0 */ ///:~
In this trivial case, the function
test( ) maintains the global variables nPass and
nFail. The only visual inspection you do is to read the final score. If a
test failed, then a more sophisticated test( ) would print out an
appropriate message. The framework described below has such a
test( ) function, among other things.
As you continue in the test-and-code
cycle you'll want to build a suite of tests that are always available to keep
all your related classes in good shape through any future maintenance. As
requirements change, you add or modify tests
accordingly.
As you learn more about XP you'll
discover that there are some automated unit test tools available for download,
such as JUnit for Java and CppUnit for C++. These are brilliantly designed and
implemented, but I want something even simpler. I want something that I can not
only easily use but also understand internally and even tweak if necessary. And
I can live without a GUI. So, in the spirit of
TheSimplestThingThatCouldPossiblyWork, I present the TestSuite Framework,
as I call it, which consists of two classes: Test and Suite. You
derive from Test (an abstract class) to override the run( )
method, which should in turn call test_ for each Boolean test condition
you define. For the Date class above you could do something like the
following:
//: C02:DateTest.h #include "../TestSuite/Test.h" #include "Date.h" class DateTest : public Test { Date mybday; Date today; public: DateTest() : mybday(1951, 10,1) {} void run() { testOps(); testDuration(); } void testOps() { test_(mybday < today); test_(mybday <= today); test_(mybday != today); test_(mybday == mybday); } void testDuration() { Date d2(2001, 7, 4); Duration dur = duration(mybday, d2); test_(dur.years == 49); test_(dur.months == 9); test_(dur.days == 3); } }; ///:~
You can now run the test very easily,
like this:
//: C02:DateTest.cpp // Automated Testing (with a Framework) //{L} Date ../TestSuite/Test #include <iostream> #include "DateTest.h" using namespace std; int main() { DateTest test; test.run(); test.report(); return test.getNumFailed(); } /* Output: Passed: 7, Failed: 0 */ ///:~
As development continues on the
Date class, you'll add other tests called from
DateTest::run( ), and then execute the main program to see if they
all pass.
The Test class uses RTTI to get
the name of your class (e.g., DateTest) for the
report[7]. The
setStream( ) method lets you specify where the output will go, and
report sends output to that stream. Test is implemented later in this
section.
In addition to test_, the
framework includes the functions succeed_ and fail_, for cases
where a Boolean test won't do. For example, a simple Stack class template
might look like this:
//: C02:Stack.h #include <cassert> #include <cstddef> #include <stdexcept> #include <string> #include <new> using std::logic_error; using std::string; using std::bad_alloc; // MS std namespace work-around #ifndef _MSC_VER using std::size_t; #endif class StackError : public logic_error { public: StackError(const string& s) : logic_error(s) {} }; template<typename T> class Stack { public: Stack(size_t) throw(StackError, bad_alloc); ~Stack(); void push(const T&) throw(StackError); T pop() throw(StackError); T top() const throw(StackError); size_t size() const; private: T* data; size_t max; size_t ptr; }; template<typename T> inline Stack<T>::~Stack() { delete [] data; max = ptr = 0; } template<typename T> inline size_t Stack<T>::size() const { return ptr; } template<typename T> Stack<T>::Stack(size_t siz) throw(StackError, bad_alloc) { if (siz == 0) throw StackError("bad size in Stack(size_t)"); data = new T[siz]; max = siz; ptr = 0; } template<typename T> void Stack<T>::push(const T& x) throw(StackError) { if (ptr == max) throw StackError("stack overflow"); assert(ptr < max); data[ptr++] = x; } template<typename T> T Stack<T>::pop() throw(StackError) { if (ptr == 0) throw StackError("stack underflow"); return data[--ptr]; } template<typename T> T Stack<T>::top() const throw(StackError) { if (ptr == 0) throw StackError("stack underflow"); return data[ptr - 1]; } ///:~
It's easy to come up with general
categories of tests for this class. Here’s the class that defines the
tests:
//: C02:StackTest.h #ifndef STACKTEST_H #define STACKTEST_H #include "Stack.h" #include "../TestSuite/Test.h" #include <iostream> using namespace std; class StackTest : public Test { enum {SIZE = 5}; Stack<int> stk; public: StackTest() : stk(SIZE){} void run(){ testUnderflow(); testPopulate(); testOverflow(); testPop(); testBadSize(); } void testBadSize(){ try { Stack<int> s(0); fail_("Bad Size"); } catch (StackError&) { succeed_(); } } void testUnderflow(){ test_(stk.size() == 0); try { stk.top(); fail_("Underflow"); } catch (StackError&) { succeed_(); } try { stk.pop(); fail_("Underflow"); } catch (StackError&) { succeed_(); } } void testPopulate(){ try { for (int i = 0; i < SIZE; ++i) stk.push(i); succeed_(); } catch (StackError&) { fail_("Populate"); } test_(stk.size() == SIZE); test_(stk.top() == SIZE-1); } void testOverflow(){ try { stk.push(SIZE); fail_("Overflow"); } catch (StackError&) { succeed_(); } } void testPop(){ for (int i = 0; i < SIZE; ++i) test_(stk.pop() == SIZE-i-1); test_(stk.size() == 0); } }; #endif // STACKTEST_H ///:~
Here’s a main( ) that
exercises the tests:
//: C02:StackTest.cpp //{L} ../TestSuite/Test #include <iostream> #include "StackTest.h" using namespace std; int main() { StackTest t; t.run(); t.report(); return t.getNumFailed(); } ///:~
To test whether exceptions are working
correctly, you have to generate an exception and call succeed_ or
fail_ explicitly, as StackTest::testBadSize( ) class
illustrates, above. Since a stack of size zero is prohibited, "success" in this
case means that a StackError exception was caught, so I have to call
succeed_ explicitly.
Real projects usually contain many
classes, so there needs to be a way to group tests together so you can just push
a single button to test the entire project. The Suite class allows you to
collect tests into a functional unit. You add a derived Test object to a
Suite with the addTest( ) method, or you can swallow an
entire existing Suite with addSuite( ). To illustrate, the following
example combines the foregoing DateTest and StackTest into a
suite. Here's an actual test run:
//: C02:SuiteExample.cpp //{L} ../TestSuite/Test ../TestSuite/Suite Date #include <iostream> #include "../TestSuite/Suite.h" #include "DateTest.h" #include "StackTest.h" using namespace std; int main() { Suite s("Date and Stack Tests"); s.addTest(new DateTest); s.addTest(new StackTest); s.run(); long nFail = s.report(); s.free(); cout << "\nTotal failures: " << nFail << endl; return nFail; } /* Output: Suite "Date and Stack Tests" ============================ Test "DateTest": Passed: 7 Failed: 0 Test "StackTest": Passed: 14 Failed: 0 ============================ Total failures: 0 */ ///:~
Suite::run calls Test::run for each of
its contained tests. Much the same thing happens for Suite::report. Individual
test results can be written to separate streams, if desired. If the test passed
to addSuite has a stream pointer assigned already, it keeps it. Otherwise it
gets its stream from the Suite object. The code for Suite is in listing 5 and 6.
As you can see, Suite just holds a vector of pointers to Test. When it's time to
run each test, it just loops through the tests in the vector calling their run
method.
It takes some discipline to write unit
tests before you code, but if you have an automated tool, it makes it a lot
easier. I just add a project in my IDE for a test suite for each project, and
switch back and forth between the test and the real code as needed. There's no
conceptual baggage, no extra test scripting language to learn, no worries
— just point, click, and
test!
The test framework code library will be
placed in a subdirectory called TestSuite. To use it, therefore, you must
have the TestSuite subdirectory in your header include search path, and
you must link the object files, and thus it must also be included in the library
search path (?? or compiled into a library ??).
Here is the header for
Test.h:
//: TestSuite:Test.h #ifndef TEST_H #define TEST_H #include <string> #include <iostream> #include <cassert> using std::string; using std::ostream; using std::cout; // The following have underscores because they // are macros (and it's impolite to usurp other // users' functions!). For consistency, // succeed_() also has an underscore. #define test_(cond) \ do_test(cond, #cond, __FILE__, __LINE__) #define fail_(str) \ do_fail(str, __FILE__, __LINE__) class Test { public: Test(ostream* osptr = &cout); virtual ~Test(){} virtual void run() = 0; long getNumPassed() const; long getNumFailed() const; const ostream* getStream() const; void setStream(ostream* osptr); void succeed_(); long report() const; virtual void reset(); protected: void do_test(bool cond, const string& lbl, const char* fname, long lineno); void do_fail(const string& lbl, const char* fname, long lineno); private: ostream* osptr; long nPass; long nFail; // Disallowed: Test(const Test&); Test& operator=(const Test&); }; inline Test::Test(ostream* osptr) { this->osptr = osptr; assert(osptr); nPass = nFail = 0; } inline long Test::getNumPassed() const { return nPass; } inline long Test::getNumFailed() const { return nFail; } inline const ostream* Test::getStream() const { return osptr; } inline void Test::setStream(ostream* osptr) { this->osptr = osptr; } inline void Test::succeed_() { ++nPass; } inline void Test::reset() { nPass = nFail = 0; } #endif // TEST_H ///:~
Explanation of code here
Here is the implmentation of
Test:
//: TestSuite:Test.cpp {O} #include "Test.h" #include <iostream> #include <typeinfo> // Visual C++ requires /GR"" using namespace std; void Test::do_test(bool cond, const std::string& lbl, const char* fname, long lineno){ if (!cond) do_fail(lbl, fname, lineno); else succeed_(); } void Test::do_fail(const std::string& lbl, const char* fname, long lineno){ ++nFail; if (osptr){ *osptr << typeid(*this).name() << "failure: (" << lbl << ") , " << fname << " (line " << lineno << ")\n"; } } long Test::report() const { if (osptr){ *osptr << "Test \"" << typeid(*this).name() << "\":\n\tPassed: " << nPass << "\tFailed: " << nFail << endl; } return nFail; } ///:~
No rocket science here. Test just
keeps track of the number of successes and failures as well as the stream where
you want Test::report( ) to print the results. test_ and
fail_ are macros so that they can include filename and line number
information available from the preprocessor.
Here is the header file for
Suite:
//: TestSuite:Suite.h #ifndef SUITE_H #define SUITE_H #include "../TestSuite/Test.h" #include <vector> #include <stdexcept> using std::vector; using std::logic_error; class TestSuiteError : public logic_error { public: TestSuiteError(const string& s = "") : logic_error(s) {} }; class Suite { public: Suite(const string& name, ostream* osptr = &cout); string getName() const; long getNumPassed() const; long getNumFailed() const; const ostream* getStream() const; void setStream(ostream* osptr); void addTest(Test* t) throw (TestSuiteError); void addSuite(const Suite&) throw(TestSuiteError); void run(); // Calls Test::run() repeatedly long report() const; void free(); // Deletes tests private: string name; ostream* osptr; vector<Test*> tests; void reset(); // Disallowed ops: Suite(const Suite&); Suite& operator=(const Suite&); }; inline Suite::Suite(const string& name, ostream* osptr) : name(name) { this->osptr = osptr; } inline string Suite::getName() const { return name; } inline const ostream* Suite::getStream() const { return osptr; } inline void Suite::setStream(ostream* osptr) { this->osptr = osptr; } #endif // SUITE_H ///:~
Explanation of code here
Here is the implementation of
Suite:
//: TestSuite:Suite.cpp {O} #include "Suite.h" #include <iostream> #include <cassert> using namespace std; void Suite::addTest(Test* t) throw(TestSuiteError) { // Make sure test has a stream: if (t == 0) throw TestSuiteError( "Null test in Suite::addTest"); else if (osptr != 0 && t->getStream() == 0) t->setStream(osptr); tests.push_back(t); t->reset(); } void Suite::addSuite(const Suite& s) throw(TestSuiteError) { for (size_t i = 0; i < s.tests.size(); ++i) addTest(s.tests[i]); } void Suite::free() { // This is not a destructor because tests // don't have to be on the heap. for (size_t i = 0; i < tests.size(); ++i) { delete tests[i]; tests[i] = 0; } } void Suite::run() { reset(); for (size_t i = 0; i < tests.size(); ++i) { assert(tests[i]); tests[i]->run(); } } long Suite::report() const { if (osptr) { long totFail = 0; *osptr << "Suite \"" << name << "\"\n======="; size_t i; for (i = 0; i < name.size(); ++i) *osptr << '='; *osptr << "=\n"; for (i = 0; i < tests.size(); ++i) { assert(tests[i]); totFail += tests[i]->report(); } *osptr << "======="; for (i = 0; i < name.size(); ++i) *osptr << '='; *osptr << "=\n"; return totFail; } else return getNumFailed(); } long Suite::getNumPassed() const { long totPass = 0; for (size_t i = 0; i < tests.size(); ++i) { assert(tests[i]); totPass += tests[i]->getNumPassed(); } return totPass; } long Suite::getNumFailed() const { long totFail = 0; for (size_t i = 0; i < tests.size(); ++i) { assert(tests[i]); totFail += tests[i]->getNumFailed(); } return totFail; } void Suite::reset() { for (size_t i = 0; i < tests.size(); ++i) { assert(tests[i]); tests[i]->reset(); } } ///:~
Explanation of code here
[5]
See Kent Beck's book, eXtreme Programming Explained: Embrace Change
(Addison-Wesley, 2000, ISBN 0-201-61641-6), or visit www.Xprogramming.com for
more information on XP. The XP theme from which this section derives its title
is DoTheSimplestThingThatCouldPossiblyWork.
[6]
The seminal work on this subject is Martin Fowler's Refactoring: Improving
the Design of Existing Code (Addison-Wesley, 2000, ISBN 0-201-48567-2). See
www.refactoring.com.
[7]
If you're using Microsoft Visual C++, you need to specify the compile option
/GR. If you don't, you'll get an access violation at run time.