Gordon’s Place

This is better. As soon as I’ve allowed myself to post something trivial, then there are a whole lot of things that I’d like to say spring to mind. Here’s one.

I’ve been working on a couple of projects recently in which I’ve made use of an xUnit style framework for unit tests. Both of the projects have mainly been backend BL code, which is the stuff I usually want to write loads of tests for, but I’ve approached the test code in two different manners.

In the first I looked at each test having to be entirely self contained. I wrote Setup routines to load a bunch of fixtures from external XML files into a variety of tables before running the test. Then the tests would run, then the teardown code would tidy out all of the tables involved in the test. This worked pretty well, I could run the test with nothing but an empty DB and the test routines, but it wasn’t the quickest. Every time the tests were run, I had table loading and tidying up to be done. It never got to the stage that it was taking too long, but I could imagine a big application with a high level of test coverage wouldn’t be able to say the same thing.

So, with the second project I tried a different approach. Instead of loading up the tables with the correct data in advance, I decided to have the setup code attempt to find appropriate test in the existing tables. If it couldn’t find the test data that it needed, then it would fail the test with a “Could not find appropriate test data”. If it could find data that met all the requirements of the test, then it would use that data as the basis of all of the subsequent tests. I wouldn’t need all the overhead of loading the fixtures, but I would have to make sure there was decent test data in place.

So, what I wanted to do was to share the wisdom I’ve gained from trying it both ways. The reason that I tried the second approach at all, was that sometimes having pre-defined test fixtures and load and tidy up routines seemed a little overkill when all I was working on was a simple update routine. I was spending more time setting up the tests than writing the code. I would concede that for this simple scenario, finding your data on the fly in the existing dev database is probably fine. But anything more complex that that quickly becomes a problem. As my latest piece of code grew in complexity, I ended up spending more and more time nursing the tests through running properly by inserting records here / deleting records there. I grew to hate the “Could not find appropriate test data” messages. And I would often have bugs in the test code. Probably more often than in the code I was trying to test. Plus this lack of a solid “known state” before running the test suite would often mean that the tests were non-repeatable, they’d run once and then the second time would fail ‘cause one of the other developers had changed the data.
So, the conclusion I’ve reached is that independent test fixtures, plus setup / teardown routines to load and clear those tables, should always be part of the test routines. I keep the fixtures as XML files and check them in with the test code. That scheme did seem a little heavy handed for simple routines, but it certainly scaled up well when the complexity increased. I’m pretty sure that if I care to check it out, this little gem will already be common knowledge in the automated testing community, but since I went to all the bother of trying the experiment, I figured I at least deserved to be able to post about it.
There you go. Two posts in a week.